absc
/
dudeswave2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Import "gun", the HTTP client. We will use it for the tests and the 

ActivityPub activities.
main
absc 2024-08-11 22:08:52 +00:00
parent 640ffe4d75
commit f60a8b1725
154 changed files with 35526 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
gun/.gitignore vendored Normal file
View File

@ -0,0 +1,11 @@
.erlang.mk
.*.plt
*.d
deps
doc/guide.pdf
doc/html
doc/man*
ebin/test
ebin/*.beam
logs
test/*.beam

13
gun/LICENSE Normal file
View File

@ -0,0 +1,13 @@
Copyright (c) 2013-2023, Loïc Hoguin <essen@ninenines.eu>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

7
gun/Makefile Normal file
View File

@ -0,0 +1,7 @@
.PHONY: all clean
all:
${MAKE} -C src
clean:
${MAKE} -C src clean

43
gun/README.asciidoc Normal file
View File

@ -0,0 +1,43 @@
= Gun
Gun is an Erlang HTTP client with support for HTTP/1.1,
HTTP/2, Websocket and more.
== Goals
Gun aims to provide an *easy to use* client compatible with
HTTP/1.1, HTTP/2 and Websocket. Gun can connect through any
combination of Socks and HTTP proxies.
Gun is *always connected*. It will maintain a permanent
connection to the server, reopening it as soon as the server
closes it, saving time for the requests that come in.
All connections are *supervised* automatically, allowing
developers to focus on writing their code without worrying.
== Sponsors
Gun was previously sponsored by
http://leo-project.net/leofs/[LeoFS Cloud Storage],
https://sameroom.io/[Sameroom],
and https://pleroma.social/[Pleroma].
== Online documentation
* https://ninenines.eu/docs/en/gun/2.1/guide[User guide]
* https://ninenines.eu/docs/en/gun/2.1/manual[Function reference]
== Offline documentation
* While still online, run `make docs`
* User guide available in `doc/` in PDF and HTML formats
* Function reference man pages available in `doc/man3/` and `doc/man7/`
* Run `make install-docs` to install man pages on your system
* Full documentation in Asciidoc available in `doc/src/`
== Getting help
* Official IRC Channel: #ninenines on irc.freenode.net
* https://github.com/ninenines/gun/issues[Issues tracker]
* https://ninenines.eu/services/[Commercial Support]

35
gun/doc/src/guide/book.asciidoc Normal file
View File

@ -0,0 +1,35 @@
// a2x: --dblatex-opts "-P latex.output.revhistory=0 -P doc.publisher.show=0 -P index.numbered=0"
// a2x: --dblatex-opts "-s gun"
// a2x: -d book --attribute tabsize=4
= Gun User Guide
= Interface
include::introduction.asciidoc[Introduction]
include::start.asciidoc[Starting and stopping]
include::protocols.asciidoc[Supported protocols]
include::connect.asciidoc[Connection]
include::http.asciidoc[Using HTTP]
include::websocket.asciidoc[Using Websocket]
= Advanced
include::internals_tls_over_tls.asciidoc[Internals: TLS over TLS]
= Additional information
include::migrating_from_2.0.asciidoc[Changes since Gun 2.0]
include::migrating_from_1.3.asciidoc[Migrating from Gun 1.3 to 2.0]
include::migrating_from_1.2.asciidoc[Migrating from Gun 1.2 to 1.3]
include::migrating_from_1.1.asciidoc[Migrating from Gun 1.1 to 1.2]
include::migrating_from_1.0.asciidoc[Migrating from Gun 1.0 to 1.1]

166
gun/doc/src/guide/connect.asciidoc Normal file
View File

@ -0,0 +1,166 @@
[[connect]]
== Connection
This chapter describes how to open, monitor and close
a connection using the Gun client.
=== Gun connections
Gun is designed with the HTTP/2 and Websocket protocols in mind.
They are built for long-running connections that allow concurrent
exchange of data, either in the form of request/responses for
HTTP/2 or in the form of messages for Websocket.
A Gun connection is an Erlang process that manages a socket to
a remote endpoint. This Gun connection is owned by a user
process that is called the _owner_ of the connection, and is
managed by the supervision tree of the `gun` application.
Any process can communicate with the Gun connection
by calling functions from the module `gun`. All functions
perform their respective operations asynchronously. The Gun
connection will send Erlang messages to the calling process
whenever needed.
When the remote endpoint closes the connection, Gun attempts
to reconnect automatically.
=== Opening a new connection
The `gun:open/2,3` function must be used to open a connection.
.Opening a connection to example.org on port 443
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443).
----
If the port given is 443, Gun will attempt to connect using
TLS. The protocol will be selected automatically using the
ALPN extension for TLS. By default Gun supports HTTP/2
and HTTP/1.1 when connecting using TLS.
For any other port, Gun will attempt to connect using
plain TCP and will use the HTTP/1.1 protocol.
The transport and protocol used can be overriden via
options. The manual documents all available options.
Options can be provided as a third argument, and take the
form of a map.
.Opening a TLS connection to example.org on port 8443
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 8443, #{transport => tls}).
----
When using TLS you may want to tweak the
http://erlang.org/doc/man/ssl_app.html#configuration[configuration]
for the `ssl` application, in particular the `session_lifetime`
and `session_cache_client_max` to limit the amount of memory
used for the TLS sessions cache.
=== Waiting for the connection to be established
When Gun successfully connects to the server, it sends a
`gun_up` message with the protocol that has been selected
for the connection.
Gun provides the functions `gun:await_up/1,2,3` that wait
for the `gun_up` message. They can optionally take a monitor
reference and/or timeout value. If no monitor is provided,
one will be created for the duration of the function call.
.Synchronous opening of a connection
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443),
{ok, Protocol} = gun:await_up(ConnPid).
----
=== Handling connection loss
When the connection is lost, Gun will send a `gun_down`
message indicating the current protocol, the reason the
connection was lost and two lists of stream references.
The first list indicates open streams that _may_ have been
processed by the server. The second list indicates open
streams that the server did not process.
=== Monitoring the connection process
Because software errors are unavoidable, it is important to
detect when the Gun process crashes. It is also important
to detect when it exits normally. Erlang provides two ways
to do that: links and monitors.
Gun leaves you the choice as to which one will be used.
However, if you use the `gun:await/2,3` or `gun:await_body/2,3`
functions, a monitor may be used for you to avoid getting
stuck waiting for a message that will never come.
If you choose to monitor yourself you can do it on a permanent
basis rather than on every message you will receive, saving
resources. Indeed, the `gun:await/3,4` and `gun:await_body/3,4`
functions both accept a monitor argument if you have one already.
.Monitoring the connection process
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443).
MRef = monitor(process, ConnPid).
----
This monitor reference can be kept and used until the connection
process exits.
.Handling `DOWN` messages
[source,erlang]
----
receive
%% Receive Gun messages here...
{'DOWN', Mref, process, ConnPid, Reason} ->
error_logger:error_msg("Oops!"),
exit(Reason)
end.
----
What to do when you receive a `DOWN` message is entirely up to you.
=== Closing the connection abruptly
The connection can be stopped abruptly at any time by calling
the `gun:close/1` function.
.Immediate closing of the connection
[source,erlang]
----
gun:close(ConnPid).
----
The process is stopped immediately without having a chance to
perform the protocol's closing handshake, if any.
//=== Closing the connection gracefully
//
//The connection can also be stopped gracefully by calling the
//`gun:shutdown/1` function.
//
//.Graceful shutdown of the connection
//[source,erlang]
//----
//gun:shutdown(ConnPid).
//----
//
//Gun will refuse any new requests or messages after you call
//this function. It will however continue to send you messages
//for existing streams until they are all completed.
//
//For example if you performed a GET request just before calling
//`gun:shutdown/1`, you will still receive the response before
//Gun closes the connection.
//
//If you set a monitor beforehand, you will receive a message
//when the connection has been closed.

8
gun/doc/src/guide/gun.sty Normal file
View File

@ -0,0 +1,8 @@
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{asciidoc-dblatex}[2012/10/24 AsciiDoc DocBook Style]
%% Just use the original package and pass the options.
\RequirePackageWithOptions{docbook}
%% Define an alias for make snippets to be compatible with source-highlighter.
\lstalias{makefile}{make}

387
gun/doc/src/guide/http.asciidoc Normal file
View File

@ -0,0 +1,387 @@
[[http]]
== HTTP
This chapter describes how to use the Gun client for
communicating with an HTTP/1.1 or HTTP/2 server.
=== Streams
Every time a request is initiated, Gun creates a _stream_.
A _stream reference_ uniquely identifies a set of request and
response and must be used to perform additional operations
with a stream or to identify its messages.
Stream references use the Erlang _reference_ data type and
are therefore unique.
Streams can be canceled at any time. This will stop any further
messages from being sent to the calling process. Depending on
its capabilities, the server will also be instructed to cancel
the request.
Canceling a stream may result in Gun dropping the connection
temporarily, to avoid uploading or downloading data that will
not be used.
.Cancelling a stream
[source,erlang]
----
gun:cancel(ConnPid, StreamRef).
----
=== Sending requests
Gun provides many convenient functions for performing common
operations, like GET, POST or DELETE. It also provides a
general purpose function in case you need other methods.
The availability of these methods on the server can vary
depending on the software used but also on a per-resource
basis.
Gun will automatically set a few headers depending on the
method used. For all methods however it will set the host
header if it has not been provided in the request arguments.
This section focuses on the act of sending a request. The
handling of responses will be explained further on.
==== GET and HEAD
Use `gun:get/2,3,4` to request a resource.
.GET "/organizations/ninenines"
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/organizations/ninenines").
----
.GET "/organizations/ninenines" with custom headers
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/organizations/ninenines", [
{<<"accept">>, "application/json"},
{<<"user-agent">>, "revolver/1.0"}
]).
----
Note that the list of headers has the field name as a binary.
The field value is iodata, which is either a binary or an
iolist.
Use `gun:head/2,3,4` if you don't need the response body.
.HEAD "/organizations/ninenines"
[source,erlang]
----
StreamRef = gun:head(ConnPid, "/organizations/ninenines").
----
.HEAD "/organizations/ninenines" with custom headers
[source,erlang]
----
StreamRef = gun:head(ConnPid, "/organizations/ninenines", [
{<<"accept">>, "application/json"},
{<<"user-agent">>, "revolver/1.0"}
]).
----
It is not possible to send a request body with a GET or HEAD
request.
==== POST, PUT and PATCH
HTTP defines three methods to create or update a resource.
POST is generally used when the resource identifier (URI) isn't known
in advance when creating a resource. POST can also be used to
replace an existing resource, although PUT is more appropriate
in that situation.
PUT creates or replaces a resource identified by the URI.
PATCH provides instructions on how to modify the resource.
Both POST and PUT send the entire resource representation in their
request body. The PATCH method can be used when this is not
desirable. The request body of a PATCH method may be a partial
representation or a list of instructions on how to update the
resource.
The functions `gun:post/4,5`, `gun:put/4,5` and `gun:patch/4,5`
take a body as their fourth argument. These functions do
not require any body-specific header to be set, although
it is always recommended to set the content-type header.
Gun will set the other headers automatically.
In this and the following examples in this section, `gun:post`
can be replaced by `gun:put` or `gun:patch` for performing
a PUT or PATCH request, respectively.
.POST "/organizations/ninenines"
[source,erlang]
----
Body = "{\"msg\": \"Hello world!\"}",
StreamRef = gun:post(ConnPid, "/organizations/ninenines", [
{<<"content-type">>, "application/json"}
], Body).
----
The functions `gun:post/3,4`, `gun:put/3,4` and `gun:patch/3,4`
do not take a body in their arguments: the body must be
provided later on using the `gun:data/4` function.
It is recommended to send the content-length header if you
know it in advance, although this is not required. If it
is not set, HTTP/1.1 will use the chunked transfer-encoding,
and HTTP/2 will continue normally as it is chunked by design.
.POST "/organizations/ninenines" with delayed body
[source,erlang]
----
Body = "{\"msg\": \"Hello world!\"}",
StreamRef = gun:post(ConnPid, "/organizations/ninenines", [
{<<"content-length">>, integer_to_binary(length(Body))},
{<<"content-type">>, "application/json"}
]),
gun:data(ConnPid, StreamRef, fin, Body).
----
The atom `fin` indicates this is the last chunk of data to
be sent. You can call the `gun:data/4` function as many
times as needed until you have sent the entire body. The
last call must use `fin` and all the previous calls must
use `nofin`. The last chunk may be empty.
.Streaming the request body
[source,erlang]
----
sendfile(ConnPid, StreamRef, Filepath) ->
{ok, IoDevice} = file:open(Filepath, [read, binary, raw]),
do_sendfile(ConnPid, StreamRef, IoDevice).
do_sendfile(ConnPid, StreamRef, IoDevice) ->
case file:read(IoDevice, 8000) of
eof ->
gun:data(ConnPid, StreamRef, fin, <<>>),
file:close(IoDevice);
{ok, Bin} ->
gun:data(ConnPid, StreamRef, nofin, Bin),
do_sendfile(ConnPid, StreamRef, IoDevice)
end.
----
==== DELETE
Use `gun:delete/2,3,4` to delete a resource.
.DELETE "/organizations/ninenines"
[source,erlang]
----
StreamRef = gun:delete(ConnPid, "/organizations/ninenines").
----
.DELETE "/organizations/ninenines" with custom headers
[source,erlang]
----
StreamRef = gun:delete(ConnPid, "/organizations/ninenines", [
{<<"user-agent">>, "revolver/1.0"}
]).
----
==== OPTIONS
Use `gun:options/2,3` to request information about a resource.
.OPTIONS "/organizations/ninenines"
[source,erlang]
----
StreamRef = gun:options(ConnPid, "/organizations/ninenines").
----
.OPTIONS "/organizations/ninenines" with custom headers
[source,erlang]
----
StreamRef = gun:options(ConnPid, "/organizations/ninenines", [
{<<"user-agent">>, "revolver/1.0"}
]).
----
You can also use this function to request information about
the server itself.
.OPTIONS "*"
[source,erlang]
----
StreamRef = gun:options(ConnPid, "*").
----
==== Requests with an arbitrary method
The functions `gun:headers/4,5` or `gun:request/5,6` can be
used to send requests with a configurable method name. It is
mostly useful when you need a method that Gun does not
understand natively.
.Example of a TRACE request
[source,erlang]
----
gun:request(ConnPid, "TRACE", "/", [
{<<"max-forwards">>, "30"}
], <<>>).
----
=== Processing responses
All data received from the server is sent to the calling
process as a message. First a `gun_response` message is sent,
followed by zero or more `gun_data` messages. If something goes wrong,
a `gun_error` message is sent instead.
The response message will inform you whether there will be
data messages following. If it contains `fin` there will be
no data messages. If it contains `nofin` then one or more data
messages will follow.
When using HTTP/2 this value is sent with the frame and simply
passed on in the message. When using HTTP/1.1 however Gun must
guess whether data will follow by looking at the response headers.
You can receive messages directly, or you can use the _await_
functions to let Gun receive them for you.
.Receiving a response using receive
[source,erlang]
----
print_body(ConnPid, MRef) ->
StreamRef = gun:get(ConnPid, "/"),
receive
{gun_response, ConnPid, StreamRef, fin, Status, Headers} ->
no_data;
{gun_response, ConnPid, StreamRef, nofin, Status, Headers} ->
receive_data(ConnPid, MRef, StreamRef);
{'DOWN', MRef, process, ConnPid, Reason} ->
error_logger:error_msg("Oops!"),
exit(Reason)
after 1000 ->
exit(timeout)
end.
receive_data(ConnPid, MRef, StreamRef) ->
receive
{gun_data, ConnPid, StreamRef, nofin, Data} ->
io:format("~s~n", [Data]),
receive_data(ConnPid, MRef, StreamRef);
{gun_data, ConnPid, StreamRef, fin, Data} ->
io:format("~s~n", [Data]);
{'DOWN', MRef, process, ConnPid, Reason} ->
error_logger:error_msg("Oops!"),
exit(Reason)
after 1000 ->
exit(timeout)
end.
----
While it may seem verbose, using messages like this has the
advantage of never locking your process, allowing you to
easily debug your code. It also allows you to start more than
one connection and concurrently perform queries on all of them
at the same time.
You can also use Gun in a synchronous manner by using the _await_
functions.
The `gun:await/2,3,4` function will wait until it receives
a response to, a pushed resource related to, or data from
the given stream.
When calling `gun:await/2,3` and not passing a monitor
reference, one is automatically created for you for the
duration of the call.
The `gun:await_body/2,3,4` works similarly, but returns the
body received. Both functions can be combined to receive the
response and its body sequentially.
.Receiving a response using await
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/"),
case gun:await(ConnPid, StreamRef) of
{response, fin, Status, Headers} ->
no_data;
{response, nofin, Status, Headers} ->
{ok, Body} = gun:await_body(ConnPid, StreamRef),
io:format("~s~n", [Body])
end.
----
=== Handling streams pushed by the server
The HTTP/2 protocol allows the server to push more than one
resource for every request. It will start sending those
extra resources before it starts sending the response itself,
so Gun will send you `gun_push` messages before `gun_response`
when that happens.
You can safely choose to ignore `gun_push` messages, or
you can handle them. If you do, you can either receive the
messages directly or use _await_ functions.
The `gun_push` message contains both the new stream reference
and the stream reference of the original request.
.Receiving a pushed response using receive
[source,erlang]
----
receive
{gun_push, ConnPid, OriginalStreamRef, PushedStreamRef,
Method, Host, Path, Headers} ->
enjoy()
end.
----
If you use the `gun:await/2,3,4` function, however, Gun
will use the original reference to identify the message but
will return a tuple that doesn't contain it.
.Receiving a pushed response using await
[source,erlang]
----
{push, PushedStreamRef, Method, URI, Headers}
= gun:await(ConnPid, OriginalStreamRef).
----
The `PushedStreamRef` variable can then be used with `gun:await/2,3,4`
and `gun:await_body/2,3,4`.
=== Flushing unwanted messages
Gun provides the function `gun:flush/1` to quickly get rid
of unwanted messages sitting in the process mailbox. You
can use it to get rid of all messages related to a connection,
or just the messages related to a stream.
.Flush all messages from a Gun connection
[source,erlang]
----
gun:flush(ConnPid).
----
.Flush all messages from a specific stream
[source,erlang]
----
gun:flush(StreamRef).
----
=== Redirecting responses to a different process
Gun allows you to specify which process will handle responses
to a request via the `reply_to` request option.
.GET "/organizations/ninenines" to a different process
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/organizations/ninenines", [],
#{reply_to => Pid}).
----

177
gun/doc/src/guide/internals_tls_over_tls.asciidoc Normal file
View File

@ -0,0 +1,177 @@
== Internals: TLS over TLS
The `ssl` application that comes with Erlang/OTP implements
TLS using an interface equivalent to the `gen_tcp` interface:
you get and manipulate a socket. The TLS encoding and
decoding is applied transparently to the data sent and
received.
In order to have a TLS layer inside another TLS layer we
need a way to encode the data of the inner layer before
we pass it to the outer layer. We cannot do this with
a socket interface. Thankfully, the `ssl` application comes
with options that allow to transform an `sslsocket()` into
an encoder/decoder.
The implementation is however a little convoluted as a
result. This chapter aims to give an overview of how it
all works under the hood.
=== gun_tls_proxy
The module `gun_tls_proxy` implements an intermediary process
that sits between the Gun process and the TLS process. It is
responsible for routing data from the Gun process to the TLS
process, and from the TLS process to the Gun process.
In order to obtain the TLS encoded data the `cb_info` option
is given to the `ssl:connect/3` function. This replaces the
default TCP outer socket module with our own custom module.
Gun uses the `gun_tls_proxy_cb` module instead. This module
will forward all messages to the `gun_tls_proxy` process.
The resulting operations looks like this:
----
Gun process <-> gun_tls_proxy <-> sslsocket() <-> gun_tls_proxy <-> "inner socket"
----
The "inner socket" is the socket for the Gun connection.
The `gun_tls_proxy` process decides where to send or
receive the data based on where it's coming from. This
is how it knows whether the data has been encoded/decoded
or not.
Because the `ssl:connect/3` function call is blocking,
a temporary process is used while connecting. This is required
because the `gun_tls_proxy` needs to forward data even while
performing the TLS handshake, otherwise the `ssl:connect/3`
call will not complete.
The result of the `ssl:connect/3` call is forward to the Gun
process, along with the negotiated protocols when the connection
was successful.
The `gun_tls_proxy_cb` module does not actually implement
`{active,N}` as requested by the `ssl` application. Instead
it uses `{active,true}`.
The behavior of the `gun_tls_proxy` process will vary depending
on whether the TLS over TLS is done connection-wide or only
stream-wide.
=== Connection-wide TLS over TLS
When used for the entire connection, the `gun_tls_proxy` process
will act like a real socket once connected. The only difference
is how the connection is performed. As mentioned earlier, the
result of the `ssl:connect/3` call is sent back to the Gun process.
When doing TLS over TLS the processes will end up looking like
this:
----
Gun process <-> gun_tls_proxy <-> "inner socket"
----
The details of the interactions between `gun_tls_proxy` and
its associated `sslsocket()` have been removed in order to
better illustrate the concept.
When adding another layer this becomes:
----
Gun process <-> gun_tls_proxy <-> gun_tls_proxy <-> sslsocket()
----
This is what is done when only HTTP/1.1 and SOCKS proxies are
involved.
=== Stream-wide TLS over TLS
The same cannot be done for HTTP/2 proxies. This is because the
HTTP/2 CONNECT method does not apply to the entire connection,
but only to a stream. The proxied data must be wrapped inside
a DATA frame. It cannot be sent directly. This is what must be
done:
----
Gun process -> gun_tls_proxy -> Gun process -> "inner socket"
----
The "inner socket" is the socket for the HTTP/2 connection.
In order to tell Gun to continue processing the data, the
`handle_continue` mechanism is introduced. When `gun_tls_proxy`
has TLS encoded the data it sends it back to the Gun process,
wrapped in a `handle_continue` tuple. This tuple contains
enough information to figure out what stream the data belongs
to and what should be done next. Gun will therefore route the
data to the correct layer and continue sending it.
This solution is also used for receiving data, except in the
reverse order.
=== Routing to the right stream
In order to know where to route the data, the `stream_ref`
had to be modified to contain all the references to the
individual streams. So if the tunnel is identified by
`StreamA` and a request on this tunnel is identified
by `StreamB`, then the stream is known as `[StreamA, StreamB]`
to the user. Gun then routes first to `StreamA`, a
tunnel, which continues routing to `StreamB`.
A problem comes up if an intermediary is a SOCKS server,
for example in the following scenario:
----
HTTP/2 proxy <-> SOCKS proxy <-> HTTP/1.1 origin
----
The SOCKS protocol doesn't have a concept of stream,
therefore when we refer to request to the origin server
they are `[StreamA, StreamB]`, not `[StreamA, StreamB, StreamC]`.
This is a problem for routing encoded/decoded TLS data
to the SOCKS layer: we don't have a built-in way of referring
to the SOCKS layer.
The solution is to have a separate `handle_continue_stream_ref`
value that assigns a reference to the SOCKS layers. Gun will
then be able to forward `handle_continue` message, and only
them, to the appropriate layer.
Gun therefore has two different routing avenues, but the
mechanism remains the same otherwise.
=== gun_tunnel
In order to simplify the routing, the `gun_tunnel` module
was introduced. For each intermediary (including the original
CONNECT stream at the HTTP/2 layer) there is a `gun_tunnel`
module.
Going back to the example above:
----
HTTP/2 proxy <-> SOCKS proxy <-> HTTP/1.1 origin
----
In this case the modules involved to handle the request
will be as follow:
----
gun_http2 <-> gun_tunnel <-> gun_tunnel <-> gun_http
----
The `gun_http2` module doesn't do any routing, it just
passes everything to the `gun_tunnel` module. The `gun_tunnel`
module will then do the routing, which involves removing
`StreamA` from `[StreamA, StreamB]` where appropriate.
The `gun_tunnel` module also receives the TLS encoded/decoded
data and forwards it appropriately. When it comes to sending
data, it will return a `send` command that allows the previous
module to continue sending the data. The `gun_http2` module
will ultimately wrap the data to be sent in a DATA frame and
send it to the "inner socket".

49
gun/doc/src/guide/introduction.asciidoc Normal file
View File

@ -0,0 +1,49 @@
[[introduction]]
== Introduction
Gun is an HTTP client for Erlang/OTP.
Gun supports the HTTP/2, HTTP/1.1 and Websocket protocols.
=== Prerequisites
Knowledge of Erlang, but also of the HTTP/1.1, HTTP/2 and Websocket
protocols is required in order to read this guide.
=== Supported platforms
Gun is tested and supported on Linux, FreeBSD, Windows and OSX.
Gun is developed for Erlang/OTP 22.0 and newer.
=== License
Gun uses the ISC License.
----
Copyright (c) 2013-2023, Loïc Hoguin <essen@ninenines.eu>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
----
=== Versioning
Gun uses http://semver.org/[Semantic Versioning 2.0.0].
=== Conventions
In the HTTP protocol, the method name is case sensitive. All standard
method names are uppercase.
Header names are case insensitive. Gun converts all the header names
to lowercase, including request headers provided by your application.

21
gun/doc/src/guide/migrating_from_1.0.asciidoc Normal file
View File

@ -0,0 +1,21 @@
[appendix]
== Migrating from Gun 1.0 to 1.1
Gun 1.1 updates the Cowlib dependency to 2.5.1 and fixes a
few problems with experimental features.
=== Features added
* Update Cowlib to 2.5.1
=== Bugs fixed
* A bug in the experimental `gun_sse_h` where lone id lines
were not propagated has been fixed by updating the Cowlib
dependency.
* The status code was incorrectly given to the experimental
content handlers as a binary. It has been fixed an an
integer is now given as was intended.
* A number of Dialyzer warnings have been fixed.

28
gun/doc/src/guide/migrating_from_1.1.asciidoc Normal file
View File

@ -0,0 +1,28 @@
[appendix]
== Migrating from Gun 1.1 to 1.2
Gun 1.2 adds support for the CONNECT request over HTTP/1.1
connections.
=== Features added
* CONNECT requests can now be issued on HTTP/1.1 connections.
The tunneled connection can use any of the protocols Gun
supports: HTTP/1.1, HTTP/2 and Websocket over both TCP and
TLS transports. Note that Gun currently does not support
tunneling a TLS connection over a TLS connection due to
limitations in Erlang/OTP.
* Gun supports sending multiple CONNECT requests, allowing
the tunnel to the origin server to go through multiple
proxies.
* Gun supports sending CONNECT requests with authorization
credentials using the Basic authentication mechanism.
* Update Cowlib to 2.6.0
=== Functions added
* The functions `gun:connect/2,3,4` have been added. They can
be used to initiate CONNECT requests on HTTP/1.1 connections.

39
gun/doc/src/guide/migrating_from_1.2.asciidoc Normal file
View File

@ -0,0 +1,39 @@
[appendix]
== Migrating from Gun 1.2 to 1.3
Gun 1.3 improves the support for CONNECT requests
introduced in the previous version and documents
Websocket protocol negotiation.
=== Features added
* The `protocols` CONNECT destination option has been added
as a replacement for the now deprecated `protocol` option.
* Add built-in support for Websocket protocol negotiation
through the Websocket option `protocols`. The interface
of the handler module currently remains undocumented and
must be set to `gun_ws_h`.
* Add the h2specd HTTP/2 test suite from the h2spec project.
=== Bugs fixed
* Fix connecting to HTTP/2 over TLS origin servers via
HTTP/1.1 CONNECT proxies.
* Do not send the HTTP/1.1 keepalive while waiting for
a response to a CONNECT request.
* Do not crash on HTTP/2 HEADERS frames with the
PRIORITY flag set.
* Do not crash on HTTP/2 HEADERS frames when the
END_HEADERS flag is not set.
* Do not crash on unknown HTTP/2 frame types.
* Reject HTTP/2 WINDOW_UPDATE frames when they would
cause the window to overflow.
* Send a GOAWAY frame on closing the HTTP/2 connection.

329
gun/doc/src/guide/migrating_from_1.3.asciidoc Normal file
View File

@ -0,0 +1,329 @@
[appendix]
== Migrating from Gun 1.3 to 2.0
Gun 2.0 includes state of the art tunnel support. With
Gun 2.0 it is possible to make requests or data go through
any number of proxy endpoints using any combination of
TCP or TLS transports and HTTP/1.1, HTTP/2 or SOCKS5
protocols. All combinations of the scenario Proxy1 ->
Proxy2 -> Origin are tested and known to work.
Gun 2.0 adds many more features such as Websocket over
HTTP/2, a built-in cookie store, graceful shutdown, flow
control for data messages, event handlers and more.
Gun 2.0 also introduces an experimental pool module that
automatically maintains connections and routes requests
to the right process, in a similar way as browsers do.
Gun 2.0 greatly improves the HTTP/2 performance when it
comes to receiving large response bodies; and when receiving
response bodies from many separate requests concurrently.
Gun now shares much of its HTTP/2 code with Cowboy,
including the HTTP/2 state machine. Numerous issues were
fixed as a result because the Cowboy implementation was
much more advanced.
The Gun connection process is now implemented using `gen_statem`.
Gun 2.0 requires Erlang/OTP 22.0 or greater.
=== Features added
* Cookie store support has been added. The `cookie_store`
option allows configuring the cookie store backend.
The `gun_cookies` module provides functions to help
implementing such a backend. Gun comes with the backend
`gun_cookies_list` which provides a per-connection,
non-persistent cookie store. The cookie store engine
implements the entire RFC6265bis draft algorithms except
the parts about non-HTTP cookies as no such interface is
provided; and the parts about SameSite as Gun has no
concept of "browsing context".
* Graceful shutdown has been implemented. Graceful shutdown
can be initiated on the client side by calling the new
function `gun:shutdown/1` or when the owner process goes
away; or on the peer side via the connection: close HTTP/1.1
header, the HTTP/2 GOAWAY frame or the Websocket close frame.
Gun will try to complete existing streams when possible;
other streams get canceled immediately. The `closing_timeout`
option controls how long we are willing to wait at most
before closing the connection.
* Gun will better detect connection failures by checking the
return value from sending data to the socket. This applies
to all supported protocols. In addition, Gun now enables
`send_timeout_close` with a `send_timeout` value defaulting
to 15s.
* Flow control has been added. It allows limiting the number
of data/Websocket messages Gun sends to the calling process.
Gun will stop reading from the socket or stop updating the
protocol's flow control window when applicable as well, to
apply some backpressure to the remote endpoint(s). It is
disabled by default and can be applied on a per-request
basis if necessary.
* An event handler interface has been added providing access
to many internal Gun events. This can be used for a variety
of purposes including logging, tracing or otherwise
instrumenting a Gun connection.
* In order to get separate events when connecting, the domain
lookup, connection and TLS handshakes are now performed
separately by Gun. As a result, there exists three separate
timeout options for each of the steps, and the transport
options had to be split into `tcp_opts` and `tls_opts`.
* Gun now supports connecting through SOCKS proxies,
including secure SOCKS proxies. Both unauthenticated
and username/password authentication are supported.
* Gun can connect through any number of HTTP, HTTPS, SOCKS
or secure SOCKS proxies, including SOCKS proxies
located after HTTP(S) proxies. The ultimate endpoint
may be using any protocol, including plain TCP, TLS,
HTTP/1.1 or HTTP/2.
* When specifying which protocols to use, options can
now be provided specific to those protocols. It is
now possible to have separate HTTP options for an
HTTP proxy and the origin HTTP server, for example.
See the new `gun:protocols()` type for details.
* Gun can now be used to send and receive raw data,
as if it was just a normal socket. This can be
useful when needing to connect through a number
of HTTP/Socks proxies, allowing the use of Gun's
great proxying capabilities (including TLS over TLS)
for any sort of protocols. This can also be useful
when performing HTTP/1.1 Upgrade to custom protocols.
* Headers can now be provided as a map.
* Header names may now be provided as binary, string or atom.
* Gun now automatically lowercases provided header names.
* Many HTTP/2 options have been added, allowing great
control over how Gun and the remote endpoint are
using the HTTP/2 connection. They can be used to
improve performance or lower the memory usage, for
example.
* A new `keepalive_tolerance` option for HTTP/2 enables
closing the connection automatically when ping acks
are not received in a timely manner. It nicely
complements the `keepalive` option that makes Gun
send pings.
* Gun now supports Websocket subprotocol negotiation
and the feature is fully documented and tested.
This can be used to create handlers that will
implement a protocol from within the Gun process itself.
The negotiation is enabled by setting the `protocols`
setting. The `default_protocol` and `user_opts`
settings are also useful.
* It is now possible to send many Websocket frames in
a single `gun:ws_send/3` call.
* Gun may now send Websocket ping frames automatically
at intervals determined by the `keepalive` option. It
is disabled by default.
* A new `silence_pings` option can be set to `false` to
receive all ping and pong frames when using Websocket.
They are typically not needed and therefore silent by
default.
* The `reply_to` option has been added to `gun:ws_upgrade/4`.
The option applies to both the response and subsequent
Websocket frames.
* The `reply_to` option is also propagated to messages
following a CONNECT request when the protocol requested
is not HTTP.
* A new option `retry_fun` can be used to implement
different backoff strategies when reconnecting.
* A new option `supervise` can be used to start a Gun
connection without using Gun's supervisor. It defaults
to `true`.
* Many improvements have been done to postpone or reject
requests and other operations while in the wrong state
(for example during state transitions when switching
protocols or connecting to proxies).
* Update Cowlib to 2.12.0.
=== Experimental features added
* The `gun_pool` module was introduced. Its interface
is very similar to the `gun` module, but as it is an
experimental feature, it has not been documented yet.
The intent is to obtain feedback and document it in
an upcoming minor release. Pools are created for each
authority (host/port) and scope (user-defined value)
pairs and are resolved accordingly using the information
provided in the request and request options. Connections
may concurrently handle multiple requests/responses
from as many different processes as required.
=== Features removed
* Gun used to reject operations by processes that were not
the owner of the connection. This behavior has been removed.
In general the caller of a request or other operation will
receive the relevant messages unless the `reply_to` option
is used.
* The `connect_destination()` option `protocol` has been
removed. It was previously deprecated in favor of `protocols`.
* The `keepalive` timeout is now disabled by default
for HTTP/1.1 and HTTP/2. To be perfectly clear, this
is unrelated to the HTTP/1.1 keep-alive mechanism.
=== Functions added
* The function `gun:set_owner/2` has been added. It allows
changing the owner of a connection process. Only the current
owner can do this operation.
* The function `gun:shutdown/1` has been added. It initiates
the graceful shutdown of the connection, followed by the
termination of the Gun process.
* The function `gun:stream_info/2` has been added. It provides
information about a specific HTTP stream.
=== Functions modified
* The function `gun:info/1` now returns the owner of the
connection as well as the cookie store.
* The functions `gun:await/2,3,4`, `gun:await_body/2,3,4` and
`gun:await_up/1,2,3` now distinguish the error types. They
can be a timeout, a connection error, a stream error or a
down error (when the Gun process exited while waiting).
* The functions `gun:await/2,3,4` will now receive upgrades,
tunnel up and Websocket messages and return them.
* Requests may now include the `tunnel` option to send the
request on a specific tunnel.
* The functions `gun:request/4,5,6` have been replaced with
`gun:headers/4,5` and `gun:request/5,6`. This provides a
cleaner separation between requests that are followed by
a body and those that aren't.
* The function `gun:ws_send/2` has been replaced with the
function `gun:ws_send/3`. The stream reference for the
corresponding Websocket upgrade request must now be given.
=== Messages added
* The `gun_tunnel_up` message has been added.
=== Messages modified
* The `gun_down` message no longer has its final element
documented as `UnprocessedStreams`. It never worked and
was always an empty list.
=== Bugs fixed
* *POTENTIAL SECURITY VULNERABILITY*: Fix transfer-encoding
precedence over content-length in responses. This bug may
contribute to a response smuggling security vulnerability
when Gun is used inside a proxy.
* Gun will now better detect connection closes in some cases.
* Gun will no longer send duplicate connection-wide `gun_error`
messages to the same process.
* Gun no longer crashes when trying to upgrade to Websocket
over a connection restricted to HTTP/1.0.
* The default value for the preferred protocols when using
CONNECT over TLS has been corrected. It was mistakenly not
enabling HTTP/2.
* Protocol options provided for a tunnel destination were
sometimes ignored. This should no longer be the case.
* Gun will no longer send an empty HTTP/2 DATA frame when
there is no request body. It was not necessary.
* Gun will no longer error out when the owner process exits.
The error reason will now be a `shutdown` tuple instead.
* The host header was set incorrectly during Websocket upgrades
when the host was configured with an IP address, resulting
in a crash. This has been corrected.
* A completed stream could be found in the `gun_down` message when
the response contained a connection: close header. This is no
longer the case.
* Hostnames can now be provided as atom as stated by the
documentation.
* Gun will no longer attempt to send empty data chunks. When
using HTTP/1.1 chunked transfer-encoding this caused the
request body to end, even when `nofin` was given.
* Gun now always retries connecting immediately when the
connection goes down.
* The default port number for the HTTP and HTTPS schemes is
no longer sent in the host header.
* An invalid stream reference was sent on failed Websocket
upgrade responses. This has been corrected.
* HTTP/2 connection preface errors are now properly detected
and propagated in the `gun_down` message to the connection
owner as well as the exit reason of the Gun process.
* HTTP/2 connection preface errors now provide a different
human readable error when the data received looks like an
HTTP/1.x response.
* HTTP/2 connection errors were missing the human readable
reason in the `gun_error` message. This has been corrected.
* Fix the host and :authority (pseudo-)headers when connecting
to an IPv6 address given as a tuple. They were lacking the
surrounding brackets.
* Fix a crash in gun:info/1 when the socket was closed before
we call Transport:sockname/1.
* Fix flushing by stream reference. When the `gun_inform`
message was flushed the function would switch to flushing
all messages from the pid instead of only messages from
the given stream.
* Allow setting a custom SNI value.
* Fix double sending of last chunk in HTTP/1.1 when Gun is
asked to send empty data before closing the stream.
* Gun will now properly ignore parameters when the media
type is text/event-stream.
* Avoid noisy crashes in the TLS over TLS code.
* Gun will now include the StreamRef of Websocket streams
when sending `gun_down` messages.
* Gun will no longer reject HTTP proxies that use HTTP/1.0
for the version in their response.

25
gun/doc/src/guide/migrating_from_2.0.asciidoc Normal file
View File

@ -0,0 +1,25 @@
[appendix]
== Migrating from Gun 2.0 to 2.1
Gun 2.1 contains a small security improvement for
the HTTP/2 protocol, as well as includes a small
number of fixes and improvements.
Gun 2.1 requires Erlang/OTP 22.0 or greater.
=== Features added
* A new HTTP/2 option `max_fragmented_header_block_size` has
been added to limit the size of header blocks that are
sent over multiple HEADERS and CONTINUATION frames.
* Update Cowlib to 2.13.0.
=== Bugs fixed
* Gun will no longer configure the NPN TLS extension,
which has long been replaced by ALPN. NPN is not
compatible with TLS 1.3.
* Gun will no longer crash when TLS connections close
very early in the connection's life time.

120
gun/doc/src/guide/protocols.asciidoc Normal file
View File

@ -0,0 +1,120 @@
[[protocols]]
== Supported protocols
This chapter describes the protocols supported and the
operations available to them.
=== HTTP/1.1
HTTP/1.1 is a text request-response protocol. The client
sends a request, the server sends back a response.
Gun provides convenience functions for performing GET, HEAD,
OPTIONS, POST, PATCH, PUT, and DELETE requests. All these
functions are aliases of `gun:headers/4,5` or `gun:request/5,6`
for the respective methods. Gun also provides a `gun:data/4`
function for streaming the request body.
Gun will send a `gun_inform` message for every intermediate
informational responses received. They will always be sent
before the `gun_response` message.
Gun will send a `gun_response` message for every response
received, followed by zero or more `gun_data` messages for
the response body, which is optionally terminated by a
`gun_trailers` message. If something goes wrong, a `gun_error`
will be sent instead.
Gun provides convenience functions for dealing with messages.
The `gun:await/2,3,4` function waits for a response to the given
request, and the `gun:await_body/2,3,4` function for the
response body. The `gun:flush/1` function can be used to clear all
messages related to a request or a connection from the mailbox
of the calling process.
The function `gun:cancel/2` can be used to silence the
response to a request previously sent if it is no longer
needed. When using HTTP/1.1 there is no multiplexing so
Gun will have to receive the response fully before any
other responses can be received.
Finally, Gun can upgrade an HTTP/1.1 connection to Websocket.
It provides the `gun:ws_upgrade/2,3,4` function for that
purpose. A `gun_upgrade` message will be sent on success;
a `gun_response` message otherwise.
=== HTTP/2
HTTP/2 is a binary protocol based on HTTP, compatible with
the HTTP semantics, that reduces the complexity of parsing
requests and responses, compresses the HTTP headers and
allows the server to push additional resources along with
the normal response to the original request.
The HTTP/2 interface is very similar to HTTP/1.1, so this
section instead focuses on the differences in the interface
for the two protocols.
Gun will send `gun_push` messages for every push received.
They will always be sent before the `gun_response` message.
They can be ignored safely if they are not needed, or they
can be canceled.
The `gun:cancel/2` function will use the HTTP/2 stream
cancellation mechanism which allows Gun to inform the
server to stop sending a response for this particular
request, saving resources.
=== Websocket
Websocket is a binary protocol built on top of HTTP that
allows asynchronous concurrent communication between the
client and the server. A Websocket server can push data to
the client at any time.
Once the Websocket connection is established over an HTTP/1.1
connection, the only operation available on this connection
is sending Websocket frames using `gun:ws_send/3`.
Gun will send a `gun_ws` message for every frame received.
=== Summary
The two following tables summarize the supported operations
and the messages Gun sends depending on the connection's
current protocol.
.Supported operations per protocol
[cols="<,3*^",options="header"]
|===
| Operation | HTTP/1.1 | HTTP/2 | Websocket
| delete | yes | yes | no
| get | yes | yes | no
| head | yes | yes | no
| options | yes | yes | no
| patch | yes | yes | no
| post | yes | yes | no
| put | yes | yes | no
| request | yes | yes | no
| data | yes | yes | no
| await | yes | yes | no
| await_body | yes | yes | no
| flush | yes | yes | no
| cancel | yes | yes | no
| ws_upgrade | yes | yes | no
| ws_send | no | no | yes
|===
.Messages sent per protocol
[cols="<,3*^",options="header"]
|===
| Message | HTTP/1.1 | HTTP/2 | Websocket
| gun_push | no | yes | no
| gun_inform | yes | yes | no
| gun_response | yes | yes | no
| gun_data | yes | yes | no
| gun_trailers | yes | yes | no
| gun_error | yes | yes | yes
| gun_upgrade | yes | yes | no
| gun_ws | no | no | yes
|===

43
gun/doc/src/guide/start.asciidoc Normal file
View File

@ -0,0 +1,43 @@
[[start]]
== Starting and stopping
This chapter describes how to start and stop the Gun application.
=== Setting up
Specify Gun as a dependency to your application in your favorite
build tool.
With Erlang.mk this is done by adding `gun` to the `DEPS` variable
in your Makefile.
.Adding Gun as an Erlang.mk dependency
[source,make]
----
DEPS = gun
----
=== Starting
Gun is an _OTP application_. It needs to be started before you can
use it.
.Starting Gun in an Erlang shell
[source,erlang]
----
1> application:ensure_all_started(gun).
{ok,[crypto,cowlib,asn1,public_key,ssl,gun]}
----
=== Stopping
You can stop Gun using the `application:stop/1` function, however
only Gun will be stopped. This is the reverse of `application:start/1`.
The `application_ensure_all_started/1` function has no equivalent for
stopping all applications.
.Stopping Gun
[source,erlang]
----
application:stop(gun).
----

124
gun/doc/src/guide/websocket.asciidoc Normal file
View File

@ -0,0 +1,124 @@
[[websocket]]
== Websocket
This chapter describes how to use the Gun client for
communicating with a Websocket server.
// @todo recovering from connection failure, reconnecting to Websocket etc.
=== HTTP upgrade
Websocket is a protocol built on top of HTTP. To use Websocket,
you must first request for the connection to be upgraded. Only
HTTP/1.1 connections can be upgraded to Websocket, so you might
need to restrict the protocol to HTTP/1.1 if you are planning
to use Websocket over TLS.
You must use the `gun:ws_upgrade/2,3,4` function to upgrade
to Websocket. This function can be called anytime after connection,
so you can send HTTP requests before upgrading to Websocket.
.Upgrade to Websocket
[source,erlang]
----
gun:ws_upgrade(ConnPid, "/websocket").
----
Gun will set all the necessary headers for performing the
Websocket upgrade, but you can specify additional headers
if needed. For example you can authenticate.
.Upgrade to Websocket using HTTP authentication
[source,erlang]
----
gun:ws_upgrade(ConnPid, "/websocket", [
{<<"authorization">>, "Basic dXNlcm5hbWU6cGFzc3dvcmQ="}
]).
----
You can pass the Websocket options as part of the `gun:open/2,3`
call when opening the connection, or using the `gun:ws_upgrade/4`.
The fourth argument is those same options.
Gun can negotiate the protocol to be used for the Websocket
connection. The `protocols` option can be given with a list
of protocols accepted and the corresponding handler module.
Note that the interface for handler modules is currently
undocumented and must be set to `gun_ws_h`.
.Upgrade to Websocket with protocol negotiation
[source,erlang]
----
StreamRef = gun:ws_upgrade(ConnPid, "/websocket", []
#{protocols => [{<<"xmpp">>, gun_ws_h}]}).
----
The upgrade will fail if the server cannot satisfy the
protocol negotiation.
When the upgrade succeeds, a `gun_upgrade` message is sent.
If the server does not understand Websocket or refused the
upgrade, a `gun_response` message is sent. If Gun couldn't
perform the upgrade due to an error (for example attempting
to upgrade to Websocket on an HTTP/1.0 connection) then a
`gun_error` message is sent.
When the server does not understand Websocket, it may send
a meaningful response which should be processed. In the
following example we however ignore it:
[source,erlang]
----
receive
{gun_upgrade, ConnPid, StreamRef, [<<"websocket">>], Headers} ->
upgrade_success(ConnPid, StreamRef);
{gun_response, ConnPid, _, _, Status, Headers} ->
exit({ws_upgrade_failed, Status, Headers});
{gun_error, ConnPid, StreamRef, Reason} ->
exit({ws_upgrade_failed, Reason})
%% More clauses here as needed.
after 1000 ->
exit(timeout)
end.
----
=== Sending data
Once the Websocket upgrade has completed successfully, you no
longer have access to functions for performing requests. You
can only send and receive Websocket messages.
Use `gun:ws_send/3` to send messages to the server.
.Send a text frame
[source,erlang]
----
gun:ws_send(ConnPid, StreamRef, {text, "Hello!"}).
----
.Send a text frame, a binary frame and then close the connection
[source,erlang]
----
gun:ws_send(ConnPid, StreamRef, [
{text, "Hello!"},
{binary, BinaryValue},
close
]).
----
Note that if you send a close frame, Gun will close the connection
cleanly but may attempt to reconnect afterwards depending on the
`retry` configuration.
=== Receiving data
Gun sends an Erlang message to the owner process for every
Websocket message it receives.
[source,erlang]
----
receive
{gun_ws, ConnPid, StreamRef, Frame} ->
handle_frame(ConnPid, StreamRef, Frame)
end.
----

639
gun/doc/src/manual/gun.asciidoc Normal file
View File

@ -0,0 +1,639 @@
= gun(3)
== Name
gun - Asynchronous HTTP client
== Description
The `gun` module provides an asynchronous interface for
connecting and communicating with Web servers over HTTP,
HTTP/2 or Websocket.
== Exports
Connection:
* link:man:gun:open(3)[gun:open(3)] - Open a connection to the given host and port
* link:man:gun:open_unix(3)[gun:open_unix(3)] - Open a connection to the given Unix domain socket
* link:man:gun:set_owner(3)[gun:set_owner(3)] - Set a new owner for the connection
* link:man:gun:shutdown(3)[gun:shutdown(3)] - Gracefully close the connection
* link:man:gun:close(3)[gun:close(3)] - Brutally close the connection
* link:man:gun:info(3)[gun:info(3)] - Obtain information about the connection
Requests:
* link:man:gun:get(3)[gun:get(3)] - Get a resource representation
* link:man:gun:head(3)[gun:head(3)] - Get headers of a resource representation
* link:man:gun:options(3)[gun:options(3)] - Query the capabilities of the server or a resource
* link:man:gun:patch(3)[gun:patch(3)] - Apply a set of changes to a resource
* link:man:gun:post(3)[gun:post(3)] - Process the enclosed representation according to a resource's own semantics
* link:man:gun:put(3)[gun:put(3)] - Create or replace a resource
* link:man:gun:delete(3)[gun:delete(3)] - Delete a resource
* link:man:gun:headers(3)[gun:headers(3)] - Initiate the given request
* link:man:gun:request(3)[gun:request(3)] - Perform the given request
* link:man:gun:data(3)[gun:data(3)] - Stream the body of a request
Proxies:
* link:man:gun:connect(3)[gun:connect(3)] - Establish a tunnel to the origin server
Messages:
* link:man:gun:await(3)[gun:await(3)] - Wait for a response
* link:man:gun:await_body(3)[gun:await_body(3)] - Wait for the complete response body
* link:man:gun:await_up(3)[gun:await_up(3)] - Wait for the connection to be up
* link:man:gun:flush(3)[gun:flush(3)] - Flush all messages related to a connection or a stream
Streams:
* link:man:gun:update_flow(3)[gun:update_flow(3)] - Update a stream's flow control value
* link:man:gun:cancel(3)[gun:cancel(3)] - Cancel the given stream
* link:man:gun:stream_info(3)[gun:stream_info(3)] - Obtain information about a stream
Websocket:
* link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)] - Upgrade to Websocket
* link:man:gun:ws_send(3)[gun:ws_send(3)] - Send Websocket frames
== Messages
Gun will inform the calling process of events asynchronously
by sending any of the following messages:
Connection:
* link:man:gun_up(3)[gun_up(3)] - The connection is up
* link:man:gun_tunnel_up(3)[gun_tunnel_up(3)] - The tunnel is up
* link:man:gun_down(3)[gun_down(3)] - The connection is down
* link:man:gun_upgrade(3)[gun_upgrade(3)] - Successful protocol upgrade
* link:man:gun_error(3)[gun_error(3)] - Stream or connection-wide error
Responses:
* link:man:gun_push(3)[gun_push(3)] - Server-initiated push
* link:man:gun_inform(3)[gun_inform(3)] - Informational response
* link:man:gun_response(3)[gun_response(3)] - Response
* link:man:gun_data(3)[gun_data(3)] - Response body
* link:man:gun_trailers(3)[gun_trailers(3)] - Response trailers
Websocket:
* link:man:gun_ws(3)[gun_ws(3)] - Websocket frame
The response messages will be sent to the process that opened
the connection by default. The `reply_to` request option can
be used to redirect request-specific messages to a different
process.
== Types
=== connect_destination()
[source,erlang]
----
connect_destination() :: #{
host := inet:hostname() | inet:ip_address(),
port := inet:port_number(),
username => iodata(),
password => iodata(),
protocols => protocols(),
transport => tcp | tls,
tls_opts => [ssl:tls_client_option()],
tls_handshake_timeout => timeout()
}
----
Destination of a CONNECT request.
The default value, if any, is given next to the option name:
host, port::
Destination hostname and port number. Mandatory.
+
Upon successful completion of the CONNECT request, Gun will
begin using these as the host and port of the origin server
for subsequent requests.
username, password::
Proxy authorization credentials. They are only sent when
both options are provided.
protocols - see below::
Ordered list of preferred protocols. Please refer to the
`protocols()` type documentation for details.
+
Defaults to `[http]` when the transport is `tcp`,
and `[http2, http]` when the transport is `tls`.
transport (tcp)::
Transport that will be used for tunneled requests.
tls_opts ([])::
Options to use for tunneled TLS connections.
tls_handshake_timeout (infinity)::
Handshake timeout for tunneled TLS connections.
=== http_opts()
[source,erlang]
----
http_opts() :: #{
closing_timeout => timeout(),
cookie_ignore_informational => boolean(),
flow => pos_integer(),
keepalive => timeout(),
transform_header_name => fun((binary()) -> binary()),
version => 'HTTP/1.1' | 'HTTP/1.0'
}
----
Configuration for the HTTP protocol.
The default value is given next to the option name:
// @todo Document content_handlers and gun_sse_h.
closing_timeout (15000)::
Time to wait before brutally closing the connection when a
graceful shutdown was requested via a call to
link:man:gun:shutdown(3)[gun:shutdown(3)].
cookie_ignore_informational (false)::
Whether cookies received inside informational responses
(1xx status code) must be ignored.
flow - see below::
The initial flow control value for all HTTP/1.1 streams.
By default flow control is disabled.
keepalive (infinity)::
Time between pings in milliseconds. Since the HTTP protocol has
no standardized way to ping the server, Gun will simply send an
empty line when the connection is idle. Gun only makes a best
effort here as servers usually have configurable limits to drop
idle connections. Disabled by default due to potential
incompatibilities.
transform_header_name - see below::
A function that will be applied to all header names before they
are sent to the server. Gun assumes that all header names are in
lower case. This function is useful if you, for example, need to
re-case header names in the event that the server incorrectly
considers the case of header names to be significant.
version (`'HTTP/1.1'`)::
HTTP version to use.
=== http2_opts()
[source,erlang]
----
http2_opts() :: #{
closing_timeout => timeout(),
cookie_ignore_informational => boolean(),
flow => pos_integer(),
keepalive => timeout(),
keepalive_tolerance => non_neg_integer(),
%% HTTP/2 state machine configuration.
connection_window_margin_size => 0..16#7fffffff,
connection_window_update_threshold => 0..16#7fffffff,
enable_connect_protocol => boolean(),
initial_connection_window_size => 65535..16#7fffffff,
initial_stream_window_size => 0..16#7fffffff,
max_concurrent_streams => non_neg_integer() | infinity,
max_connection_window_size => 0..16#7fffffff,
max_decode_table_size => non_neg_integer(),
max_encode_table_size => non_neg_integer(),
max_fragmented_header_block_size => 16384..16#7fffffff,
max_frame_size_received => 16384..16777215,
max_frame_size_sent => 16384..16777215 | infinity,
max_stream_window_size => 0..16#7fffffff,
preface_timeout => timeout(),
settings_timeout => timeout(),
stream_window_data_threshold => 0..16#7fffffff,
stream_window_margin_size => 0..16#7fffffff,
stream_window_update_threshold => 0..16#7fffffff
}
----
Configuration for the HTTP/2 protocol.
The default value is given next to the option name:
// @todo Document content_handlers and gun_sse_h.
closing_timeout (15000)::
Time to wait before brutally closing the connection when a
graceful shutdown was requested either via a call to
link:man:gun:shutdown(3)[gun:shutdown(3)] or by the server.
cookie_ignore_informational (false)::
Whether cookies received inside informational responses
(1xx status code) must be ignored.
flow - see below::
The initial flow control value for all HTTP/2 streams.
By default flow control is disabled.
keepalive (infinity)::
Time between pings in milliseconds.
keepalive_tolerance - see below::
The number of unacknowledged pings in flight that are
tolerated before the connection is closed. By default
this mechanism is disabled even if `keepalive` is
enabled.
=== opts()
[source,erlang]
----
opts() :: #{
connect_timeout => timeout(),
cookie_store => gun_cookies:store(),
domain_lookup_timeout => timeout(),
http_opts => http_opts(),
http2_opts => http2_opts(),
protocols => protocols(),
retry => non_neg_integer(),
retry_fun => fun(),
retry_timeout => pos_integer(),
supervise => boolean(),
tcp_opts => [gen_tcp:connect_option()],
tls_handshake_timeout => timeout(),
tls_opts => [ssl:tls_client_option()],
trace => boolean(),
transport => tcp | tls,
ws_opts => ws_opts()
}
----
Configuration for the connection.
The default value is given next to the option name:
connect_timeout (infinity)::
Connection timeout.
cookie_store - see below::
The cookie store that Gun will use for this connection.
When configured, Gun will query the store for cookies
and include them in the request headers; and add cookies
found in response headers to the store.
+
By default no cookie store will be used.
domain_lookup_timeout (infinity)::
Domain lookup timeout.
http_opts (#{})::
Options specific to the HTTP protocol.
http2_opts (#{})::
Options specific to the HTTP/2 protocol.
protocols - see below::
Ordered list of preferred protocols. Please refer to the
`protocols()` type documentation for details.
+
Defaults to `[http]` when the transport is `tcp`,
and `[http2, http]` when the transport is `tls`.
retry (5)::
Number of times Gun will try to reconnect on failure before giving up.
retry_fun - see below::
A fun that will be called before every reconnect attempt. It receives
the current number of retries left and the Gun options. It returns the
next number of retries left and the timeout to apply before reconnecting.
The default fun will remove one to the number of retries and set the
timeout to the `retry_timeout` value.
The fun must be defined as follow:
[source,erlang]
----
fun ((non_neg_integer(), opts()) -> #{
retries => non_neg_integer(),
timeout => pos_integer()
})
----
The fun will never be called when the `retry` option is set to 0. When
this function returns 0 in the `retries` value, Gun will do one last
reconnect attempt before giving up.
retry_timeout (5000)::
Time between retries in milliseconds.
supervise (true)::
Whether the Gun process should be started under the `gun_sup`
supervisor. Set to `false` to use your own supervisor.
tcp_opts (DefaultOpts)::
TCP options used when establishing the connection.
By default Gun enables send timeouts with the options
`[{send_timeout, 15000}, {send_timeout_close, true}]`.
tls_handshake_timeout (infinity)::
TLS handshake timeout.
tls_opts ([])::
TLS options used for the TLS handshake after the connection
has been established, when the transport is set to `tls`.
trace (false)::
Whether to enable `dbg` tracing of the connection process. Should
only be used during debugging.
transport - see below::
Whether to use TLS or plain TCP. The default varies depending on the
port used. Port 443 defaults to `tls`. All other ports default to `tcp`.
ws_opts (#{})::
Options specific to the Websocket protocol.
=== protocols()
[source,erlang]
----
Protocol :: http | {http, http_opts()}
| http2 | {http2, http2_opts()}
| raw | {raw, raw_opts()}
| socks | {socks, socks_opts()}
protocols() :: [Protocol]
----
Ordered list of preferred protocols. When the transport is `tcp`,
this list must contain exactly one protocol. When the transport
is `tls`, this list must contain at least one protocol and will be
used to negotiate a protocol via ALPN. When the server does not
support ALPN then `http` will be used, except when the list of
preferred protocols is set to only accept `socks`.
Defaults to `[http]` when the transport is `tcp`,
and `[http2, http]` when the transport is `tls`.
=== raw_opts()
[source,erlang]
----
raw_opts() :: #{
}
----
Configuration for the "raw" protocol.
// The default value is given next to the option name:
=== req_headers()
[source,erlang]
----
req_headers() :: [{binary() | string() | atom(), iodata()}]
| #{binary() | string() | atom() => iodata()}
----
Request headers.
=== req_opts()
[source,erlang]
----
req_opts() :: #{
flow => pos_integer(),
reply_to => pid()
}
----
Configuration for a particular request.
The default value is given next to the option name:
flow - see below::
The initial flow control value for the stream. By default
flow control is disabled.
reply_to (`self()`)::
The pid of the process that will receive the response messages.
=== socks_opts()
[source,erlang]
----
socks_opts() :: #{
host := inet:hostname() | inet:ip_address(),
port := inet:port_number(),
auth => [{username_password, binary(), binary()} | none],
protocols => protocols(),
transport => tcp | tls,
version => 5,
tls_opts => [ssl:tls_client_option()],
tls_handshake_timeout => timeout()
}
----
Configuration for the Socks protocol.
The default value, if any, is given next to the option name:
host, port::
Destination hostname and port number. Mandatory.
+
Upon successful completion of the Socks connection, Gun will
begin using these as the host and port of the origin server
for subsequent requests.
auth ([none])::
Authentication methods Gun advertises to the Socks proxy.
protocols - see below::
Ordered list of preferred protocols. Please refer to the
`protocols()` type documentation for details.
+
Defaults to `[http]` when the transport is `tcp`,
and `[http2, http]` when the transport is `tls`.
transport (tcp)::
Transport that will be used for tunneled requests.
tls_opts ([])::
Options to use for tunneled TLS connections.
tls_handshake_timeout (infinity)::
Handshake timeout for tunneled TLS connections.
version (5)::
Version of the Socks protocol to use.
=== stream_ref()
[source,erlang]
----
stream_ref() - see below
----
Unique identifier for a stream.
The exact type is considered to be an implementation
detail.
=== ws_opts()
[source,erlang]
----
ws_opts() :: #{
closing_timeout => timeout(),
compress => boolean(),
default_protocol => module(),
flow => pos_integer(),
keepalive => timeout(),
protocols => [{binary(), module()}],
silence_pings => boolean(),
user_opts => any()
}
----
Configuration for the Websocket protocol.
The default value is given next to the option name:
closing_timeout (15000)::
Time to wait before brutally closing the connection when a
graceful shutdown was requested either via a call to
link:man:gun:shutdown(3)[gun:shutdown(3)] or by the server.
compress (false)::
Whether to enable permessage-deflate compression. This does
not guarantee that compression will be used as it is the
server that ultimately decides. Defaults to false.
default_protocol (gun_ws_h)::
Default protocol module when no Websocket subprotocol is
negotiated.
flow - see below::
The initial flow control value for the Websocket connection.
By default flow control is disabled.
keepalive (infinity)::
Time between pings in milliseconds.
protocols ([])::
A non-empty list enables Websocket protocol negotiation. The
list of protocols will be sent in the sec-websocket-protocol
request header. The given module must follow the
link:man:gun_ws_protocol(3)[gun_ws_protocol(3)] interface.
Gun comes with a default interface in `gun_ws_h` that may
be reused for negotiated protocols.
silence_pings (true)::
Whether the ping and pong frames should be sent to the user.
In all cases Gun will automatically send a pong frame back
when receiving a ping.
user_opts - see below::
Additional options that are not in use by Gun unless a custom
Websocket subprotocol is configured and negotiated.
By default no user option is defined.
== Changelog
* *2.1*: The HTTP/2 option list was updated with new options.
* *2.0*: The `default_protocol` and `user_opts` Websocket
options were added.
* *2.0*: The `stream_ref()` type was added.
* *2.0*: The option `cookie_store` was added. It can be used
to configure a cookie store that Gun will use
automatically. A related option, `cookie_ignore_informational`,
was added to both `http_opts()` and `http2_opts()`.
* *2.0*: The types `protocols()` and `socks_opts()` have been
added. Support for the Socks protocol has been added
in every places where protocol selection is available.
In addition it is now possible to specify separate
HTTP options for the CONNECT proxy and the origin server.
* *2.0*: The `connect_timeout` option has been split into
three options: `domain_lookup_timeout`, `connect_timeout`
and when applicable `tls_handshake_timeout`.
* *2.0*: The option `retry_fun` was added. It can be used to
implement different reconnect strategies.
* *2.0*: The `transport_opts` option has been split into
two options: `tcp_opts` and `tls_opts`.
* *2.0*: Function `gun:update_flow/3` introduced. The `flow`
option was added to request options and HTTP/1.1,
HTTP/2 and Websocket options as well.
* *2.0*: Introduce the type `req_headers()` and extend the
types accepted for header names for greater
interoperability. Header names are automatically
lowercased as well.
* *2.0*: Function `gun:headers/4,5` introduced.
* *2.0*: The `keepalive` option is now set to `infinity` by
default for all protocols. This means it is disabled.
* *2.0*: Websocket options `keepalive` and `silence_pings` introduced.
* *2.0*: Remove the `protocol` option from `connect_destination()`.
* *1.3*: Add the CONNECT destination's `protocols` option and
deprecate the previously introduced `protocol` option.
* *1.2*: Introduce the type `connect_destination()`.
== See also
link:man:gun(7)[gun(7)]

122
gun/doc/src/manual/gun.await.asciidoc Normal file
View File

@ -0,0 +1,122 @@
= gun:await(3)
== Name
gun:await - Wait for a response
== Description
[source,erlang]
----
await(ConnPid, StreamRef)
-> await(ConnPid, StreamRef, 5000, MonitorRef)
await(ConnPid, StreamRef, MonitorRef)
-> await(ConnPid, StreamRef, 5000, MonitorRef)
await(ConnPid, StreamRef, Timeout)
-> await(ConnPid, StreamRef, Timeout, MonitorRef)
await(ConnPid, StreamRef, Timeout, MonitorRef)
-> Result
ConnPid :: pid()
StreamRef :: gun:stream_ref()
MonitorRef :: reference()
Timeout :: timeout()
Result :: tuple() - see below
----
Wait for a response.
This function waits for a message from the given stream and
returns it as a tuple. An error will be returned should the
process fail or a relevant message is not received within
the specified duration.
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
Timeout::
How long to wait for a message, in milliseconds.
MonitorRef::
Monitor for the Gun connection process.
+
A monitor is automatically created for the duration of this
call when one is not provided.
== Return value
A number of different tuples can be returned. They correspond
to the message of the same name and they contain the same
elements minus the pid and stream reference. Error tuples
may also be returned when a timeout or an error occur.
[source,erlang]
----
Result :: {inform, Status, Headers}
| {response, IsFin, Status, Headers}
| {data, IsFin, Data}
| {trailers, Trailers}
| {push, NewStreamRef, Method, URI, Headers}
| {upgrade, Protocols, Headers}
| {ws, Frame}
| {error, Reason}
Reason :: {stream_error | connection_error | down, any()}
| timeout
----
Because the messages and returned tuples are equivalent,
please refer to the manual pages for each message for
further information:
* link:man:gun_inform(3)[gun_inform(3)] - Informational response
* link:man:gun_response(3)[gun_response(3)] - Response
* link:man:gun_data(3)[gun_data(3)] - Response body
* link:man:gun_trailers(3)[gun_trailers(3)] - Response trailers
* link:man:gun_push(3)[gun_push(3)] - Server-initiated push
* link:man:gun_upgrade(3)[gun_upgrade(3)] - Successful protocol upgrade
* link:man:gun_ws(3)[gun_ws(3)] - Websocket frame
== Changelog
* *2.0*: `upgrade` and `ws` tuples can now be returned.
* *2.0*: The error tuple type now includes the type of error.
* *1.0*: Function introduced.
== Examples
.Wait for a response
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/articles", [
{<<"accept">>, <<"text/html;q=1.0, application/xml;q=0.1">>}
]).
{response, nofin, 200, _Headers} = gun:await(ConnPid, StreamRef).
{data, fin, <<"Hello world!">>} = gun:await(ConnPid, StreamRef).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:await_body(3)[gun:await_body(3)]

91
gun/doc/src/manual/gun.await_body.asciidoc Normal file
View File

@ -0,0 +1,91 @@
= gun:await_body(3)
== Name
gun:await_body - Wait for the complete response body
== Description
[source,erlang]
----
await_body(ConnPid, StreamRef)
-> await_body(ConnPid, StreamRef, 5000, MonitorRef)
await_body(ConnPid, StreamRef, MonitorRef)
-> await_body(ConnPid, StreamRef, 5000, MonitorRef)
await_body(ConnPid, StreamRef, Timeout)
-> await_body(ConnPid, StreamRef, Timeout, MonitorRef)
await_body(ConnPid, StreamRef, Timeout, MonitorRef)
-> {ok, Body} | {ok, Body, Trailers} | {error, Reason}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
MonitorRef :: reference()
Timeout :: timeout()
Body :: binary()
Trailers :: [{binary(), binary()}]
Reason :: {stream_error | connection_error | down, any()}
| timeout
----
Wait for the complete response body.
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
Timeout::
How long to wait for each message, in milliseconds.
MonitorRef::
Monitor for the Gun connection process.
+
A monitor is automatically created for the duration of this
call when one is not provided.
== Return value
The body is returned, possibly with trailers if the
request contained a `te: trailers` header. Error tuples
may also be returned when a timeout or an error occur.
== Changelog
* *2.0*: The error tuple type now includes the type of error.
* *1.0*: Function introduced.
== Examples
.Wait for the complete response body
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/articles", [
{<<"accept">>, <<"text/html;q=1.0, application/xml;q=0.1">>}
]).
{response, nofin, 200, _Headers} = gun:await(ConnPid, StreamRef).
{ok, _Body} = gun:await_body(ConnPid, StreamRef).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:await(3)[gun:await(3)]

75
gun/doc/src/manual/gun.await_up.asciidoc Normal file
View File

@ -0,0 +1,75 @@
= gun:await_up(3)
== Name
gun:await_up - Wait for the connection to be up
== Description
[source,erlang]
----
await_up(ConnPid)
-> await_up(ConnPid, 5000, MonitorRef)
await_up(ConnPid, MonitorRef)
-> await_up(ConnPid, 5000, MonitorRef)
await_up(ConnPid, Timeout)
-> await_up(ConnPid, Timeout, MonitorRef)
await_up(ConnPid, Timeout, MonitorRef)
-> {ok, Protocol} | {error, Reason}
ConnPid :: pid()
MonitorRef :: reference()
Timeout :: timeout()
Protocol :: http | http2 | socks
Reason :: {down, any()} | timeout
----
Wait for the connection to be up.
== Arguments
ConnPid::
The pid of the Gun connection process.
Timeout::
How long to wait for, in milliseconds.
MonitorRef::
Monitor for the Gun connection process.
+
A monitor is automatically created for the duration of this
call when one is not provided.
== Return value
The protocol selected for this connection. It can be used
to determine the capabilities of the server. Error tuples
may also be returned when a timeout or an error occur.
== Changelog
* *2.0*: The error tuple type now includes the type of error.
* *1.0*: Function introduced.
== Examples
.Wait for the connection to be up
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443).
{ok, _} = gun:await_up(ConnPid).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_up(3)[gun_up(3)]

70
gun/doc/src/manual/gun.cancel.asciidoc Normal file
View File

@ -0,0 +1,70 @@
= gun:cancel(3)
== Name
gun:cancel - Cancel the given stream
== Description
[source,erlang]
----
cancel(ConnPid, StreamRef) -> ok
ConnPid :: pid()
StreamRef :: gun:stream_ref()
----
Cancel the given stream.
The behavior of this function depends on the protocol
selected.
HTTP/1.1 does not support this feature. Gun will simply
silence the stream and stop relaying messages. Gun may
also decide to close the connection if the response body
is too large, to avoid wasting time and bandwidth.
HTTP/2 allows cancelling streams at any time.
This function is asynchronous. Messages related to this
stream may still be sent after the function returns.
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
== Return value
The atom `ok` is returned.
== Changelog
* *1.0*: Function introduced.
== Examples
.Cancel a stream
[source,erlang]
----
gun:cancel(ConnPid, StreamRef).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:stream_info(3)[gun:stream_info(3)]

45
gun/doc/src/manual/gun.close.asciidoc Normal file
View File

@ -0,0 +1,45 @@
= gun:close(3)
== Name
gun:close - Brutally close the connection
== Description
[source,erlang]
----
close(ConnPid) -> ok
ConnPid :: pid()
----
Brutally close the connection.
== Arguments
ConnPid::
The pid of the Gun connection process.
== Return value
The atom `ok` is returned.
== Changelog
* *1.0*: Function introduced.
== Examples
.Close the connection
[source,erlang]
----
ok = gun:close(ConnPid).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:shutdown(3)[gun:shutdown(3)]

115
gun/doc/src/manual/gun.connect.asciidoc Normal file
View File

@ -0,0 +1,115 @@
= gun:connect(3)
== Name
gun:connect - Establish a tunnel to the origin server
== Description
[source,erlang]
----
connect(ConnPid, Destination)
-> connect(ConnPid, Destination, [], #{}).
connect(ConnPid, Destination, Headers)
-> connect(ConnPid, Destination, Headers, #{}).
connect(ConnPid, Destination, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Destination :: gun:connect_destination()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Establish a tunnel to the origin server.
This feature is currently only available for HTTP/1.1 connections.
Upon successful completion of the CONNECT request a tunnel is
established and subsequent requests will go through the tunnel.
Gun will not automatically re-issue the CONNECT request upon
reconnection to the proxy server. The `gun_up` message can
be used to know when the tunnel needs to be established again.
== Arguments
ConnPid::
The pid of the Gun connection process.
Destination::
Destination of the CONNECT request.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.2*: Function introduced.
== Examples
.Establish a tunnel
[source,erlang]
----
{ok, ConnPid} = gun:open("proxy.example.org", 1080),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:connect(ConnPid, #{
host => "origin-server.example.org",
port => 80
}),
{response, fin, 200, _} = gun:await(ConnPid, StreamRef),
%% Subsequent requests will be sent to origin-server.example.org.
----
.Establish a tunnel for a secure HTTP/2 connection
[source,erlang]
----
{ok, ConnPid} = gun:open("proxy.example.org", 1080),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:connect(ConnPid, #{
host => "origin-server.example.org",
port => 443,
protocols => [http2],
transport => tls
}),
{response, fin, 200, _} = gun:await(ConnPid, StreamRef),
%% Subsequent requests will be sent to origin-server.example.org.
----
.Establish a tunnel using proxy authorization
[source,erlang]
----
{ok, ConnPid} = gun:open("proxy.example.org", 1080),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:connect(ConnPid, #{
host => "origin-server.example.org",
port => 80,
username => "essen",
password => "myrealpasswordis"
}),
{response, fin, 200, _} = gun:await(ConnPid, StreamRef),
%% Subsequent requests will be sent to origin-server.example.org.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:await_up(3)[gun:await_up(3)],
link:man:gun_up(3)[gun_up(3)]

75
gun/doc/src/manual/gun.data.asciidoc Normal file
View File

@ -0,0 +1,75 @@
= gun:data(3)
== Name
gun:data - Stream the body of a request
== Description
[source,erlang]
----
data(ConnPid, StreamRef, IsFin, Data) -> ok
ConnPid :: pid()
StreamRef :: gun:stream_ref()
IsFin :: fin | nofin
Data :: iodata()
----
Stream the body of a request.
This function can only be used if the original request
had headers indicating that a body would be streamed.
All calls to this function must use the `nofin` flag
except for the last which must use `fin` to indicate
the end of the request body.
Empty data is allowed regardless of the value of `IsFin`.
Gun may or may not send empty data chunks, however.
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
IsFin::
Whether this message terminates the request.
Data::
All or part of the response body.
== Return value
The atom `ok` is returned.
== Changelog
* *1.0*: Function introduced.
== Examples
.Stream the body of a request
[source,erlang]
----
StreamRef = gun:put(ConnPid, "/lang/fr_FR/hello", [
{<<"content-type">>, <<"text/plain">>}
]).
gun:data(ConnPid, StreamRef, nofin, <<"Bonjour !\n">>).
gun:data(ConnPid, StreamRef, fin, <<"Bonsoir !\n">>).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:headers(3)[gun:headers(3)]

82
gun/doc/src/manual/gun.delete.asciidoc Normal file
View File

@ -0,0 +1,82 @@
= gun:delete(3)
== Name
gun:delete - Delete a resource
== Description
[source,erlang]
----
delete(ConnPid, Path)
-> delete(ConnPid, Path, [], #{}).
delete(ConnPid, Path, Headers)
-> delete(ConnPid, Path, Headers, #{})
delete(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Delete a resource.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.0*: Function introduced.
== Examples
.Delete a resource
[source,erlang]
----
StreamRef = gun:delete(ConnPid, "/drafts/123").
----
.Delete a resource with request options
[source,erlang]
----
StreamRef = gun:delete(ConnPid, "/drafts/123", [],
#{reply_to => ReplyToPid}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

59
gun/doc/src/manual/gun.flush.asciidoc Normal file
View File

@ -0,0 +1,59 @@
= gun:flush(3)
== Name
gun:flush - Flush all messages related to a connection or a stream
== Description
[source,erlang]
----
flush(ConnPid) -> ok
flush(StreamRef) -> ok
ConnPid :: pid()
StreamRef :: gun:stream_ref()
----
Flush all messages related to a connection or a stream.
== Arguments
Either of these arguments may be provided:
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
== Return value
The atom `ok` is returned.
== Changelog
* *1.0*: Function introduced.
== Examples
.Flush all messages from a connection
[source,erlang]
----
gun:flush(ConnPid).
----
.Flush messages from a single stream
[source,erlang]
----
gun:flush(StreamRef).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun:await_up(3)[gun:await_up(3)]

85
gun/doc/src/manual/gun.get.asciidoc Normal file
View File

@ -0,0 +1,85 @@
= gun:get(3)
== Name
gun:get - Get a resource representation
== Description
[source,erlang]
----
get(ConnPid, Path)
-> get(ConnPid, Path, [], #{}).
get(ConnPid, Path, Headers)
-> get(ConnPid, Path, Headers, #{})
get(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Get a resource representation.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.0*: Function introduced.
== Examples
.Get a resource representation
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/articles", [
{<<"accept">>, <<"text/html;q=1.0, application/xml;q=0.1">>}
]).
----
.Get a resource representation with request options
[source,erlang]
----
StreamRef = gun:get(ConnPid, "/articles", [], #{
reply_to => ReplyToPid
}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

92
gun/doc/src/manual/gun.head.asciidoc Normal file
View File

@ -0,0 +1,92 @@
= gun:head(3)
== Name
gun:head - Get headers of a resource representation
== Description
[source,erlang]
----
head(ConnPid, Path)
-> head(ConnPid, Path, [], #{}).
head(ConnPid, Path, Headers)
-> head(ConnPid, Path, Headers, #{})
head(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Get headers of a resource representation.
This function performs the same operation as
link:man:gun:get(3)[gun:get(3)], except the server will not
send the resource representation, only the response's status
code and headers.
While servers are supposed to send the same headers as for
a GET request, they sometimes will not. For example the
`content-length` header may be dropped from the response.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.0*: Function introduced.
== Examples
.Get headers of a resource representation
[source,erlang]
----
StreamRef = gun:head(ConnPid, "/articles", [
{<<"accept">>, <<"text/html;q=1.0, application/xml;q=0.1">>}
]).
----
.Get headers of a resource representation with request options
[source,erlang]
----
StreamRef = gun:head(ConnPid, "/articles", [], #{
reply_to => ReplyToPid
}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:head(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)]

86
gun/doc/src/manual/gun.headers.asciidoc Normal file
View File

@ -0,0 +1,86 @@
= gun:headers(3)
== Name
gun:headers - Initiate the given request
== Description
[source,erlang]
----
headers(ConnPid, Method, Path, Headers)
-> headers(ConnPid, Method, Path, Headers, #{})
headers(ConnPid, Method, Path, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Method :: binary()
Path :: iodata()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Initiate the given request.
This is a general purpose function that should only be
used when other method-specific functions do not apply.
The function `headers/4,5` initiates a request but does
not send the request body. It must be sent separately
using link:man:gun:data(3)[gun:data(3)].
== Arguments
ConnPid::
The pid of the Gun connection process.
Method::
Method to be used for the request.
Path::
Path to the resource.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *2.0*: Function introduced.
== Examples
.Initiate a request
[source,erlang]
----
StreamRef = gun:headers(ConnPid, <<"PUT">>,
"/lang/fr_FR/hello",
[{<<"content-type">>, <<"text/plain">>}]).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

69
gun/doc/src/manual/gun.info.asciidoc Normal file
View File

@ -0,0 +1,69 @@
= gun:info(3)
== Name
gun:info - Obtain information about the connection
== Description
[source,erlang]
----
info(ConnPid) -> Info
ConnPid :: pid()
Info :: #{
owner => pid(),
socket => inet:socket() | ssl:sslsocket(),
transport => tcp | tls,
protocol => http | http2 | socks | ws,
sock_ip => inet:ip_address(),
sock_port => inet:port_number(),
origin_scheme => binary() | undefined,
origin_host => inet:hostname() | inet:ip_address(),
origin_port => inet:port_number(),
intermediaries => [Intermediary],
cookie_store => gun_cookies:cookie_store()
}
Intermediary :: #{
type => connect | socks5,
host => inet:hostname() | inet:ip_address(),
port => inet:port_number(),
transport => tcp | tls,
protocol => http | http2 | socks | raw
}
----
Obtain information about the connection.
== Arguments
ConnPid::
The pid of the Gun connection process.
== Return value
A map is returned containing various informations about
the connection.
== Changelog
* *2.0*: The values `owner`, `origin_scheme` and `cookie_store` were
added.
* *1.3*: The values `socket`, `transport`, `protocol`, `origin_host`,
`origin_port` and `intermediaries` were added.
* *1.0*: Function introduced.
== Examples
.Obtain information about the connection
[source,erlang]
----
Info = gun:info(ConnPid).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)]

77
gun/doc/src/manual/gun.open.asciidoc Normal file
View File

@ -0,0 +1,77 @@
= gun:open(3)
== Name
gun:open - Open a connection to the given host and port
== Description
[source,erlang]
----
open(Host, Port) -> open(Host, Port, #{})
open(Host, Port, Opts) -> {ok, pid()} | {error, Reason}
Host :: inet:hostname() | inet:ip_address()
Port :: inet:port_number()
Opts :: gun:opts()
Reason :: {options, OptName}
| {options, {http | http2 | socks | ws, OptName}}
| any()
OptName :: atom()
----
Open a connection to the given host and port.
== Arguments
Host::
Host or IP address to connect to.
Port::
Port to connect to.
Opts::
Options for this connection.
== Return value
The pid of the newly created Gun process is returned.
Note that this does not indicate that the connection
has been successfully opened; the link:man:gun_up(3)[gun_up(3)]
message will be sent for that.
== Changelog
* *1.0*: Function introduced.
== Examples
.Connect to a server
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443).
----
.Connect to a server with custom options
[source,erlang]
----
{ok, ConnPid} = gun:open("example.org", 443,
#{protocols => [http2]}).
----
.Connect to a server using its IP address
[source,erlang]
----
{ok, ConnPid} = gun:open({127,0,0,1}, 443).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:await_up(3)[gun:await_up(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_up(3)[gun_up(3)]

65
gun/doc/src/manual/gun.open_unix.asciidoc Normal file
View File

@ -0,0 +1,65 @@
= gun:open_unix(3)
== Name
gun:open_unix - Open a connection to the given Unix domain socket
== Description
[source,erlang]
----
open_unix(SocketPath, Opts) -> {ok, pid()} | {error, Reason}
SocketPath :: string()
Opts :: gun:opts()
Reason :: {options, OptName}
| {options, {http | http2 | socks | ws, OptName}}
| any()
OptName :: atom()
----
Open a connection to the given Unix domain socket.
== Arguments
SocketPath::
Path to the Unix domain socket to connect to.
Opts::
Options for this connection.
== Return value
The pid of the newly created Gun process is returned.
Note that this does not indicate that the connection
has been successfully opened; the link:man:gun_up(3)[gun_up(3)]
message will be sent for that.
== Changelog
* *1.0*: Function introduced.
== Examples
.Connect to a server via a Unix domain socket
[source,erlang]
----
{ok, ConnPid} = gun:open_unix("/var/run/dbus/system_bus_socket", #{}).
----
.Connect to a server via a Unix domain socket with custom options
[source,erlang]
----
{ok, ConnPid} = gun:open_unix("/var/run/dbus/system_bus_socket",
#{protocols => [http2]}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:await_up(3)[gun:await_up(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_up(3)[gun_up(3)]

83
gun/doc/src/manual/gun.options.asciidoc Normal file
View File

@ -0,0 +1,83 @@
= gun:options(3)
== Name
gun:options - Query the capabilities of the server or a resource
== Description
[source,erlang]
----
options(ConnPid, Path)
-> options(ConnPid, Path, [], #{}).
options(ConnPid, Path, Headers)
-> options(ConnPid, Path, Headers, #{})
options(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Query the capabilities of the server or a resource.
The special path `"*"` can be used to obtain information about
the server as a whole. Any other path will return information
about that resource specifically.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.0*: Function introduced.
== Examples
.Query the capabilities of the server
[source,erlang]
----
StreamRef = gun:options(ConnPid, "*").
----
.Query the capabilities of a resource
[source,erlang]
----
StreamRef = gun:options(ConnPid, "/articles").
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

123
gun/doc/src/manual/gun.patch.asciidoc Normal file
View File

@ -0,0 +1,123 @@
= gun:patch(3)
== Name
gun:patch - Apply a set of changes to a resource
== Description
[source,erlang]
----
patch(ConnPid, Path, Headers)
-> patch(ConnPid, Path, Headers, #{})
patch(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
patch(ConnPid, Path, Headers, Body)
-> patch(ConnPid, Path, Headers, Body, #{})
patch(ConnPid, Path, Headers, Body, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
Body :: iodata()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Apply a set of changes to a resource.
The behavior of this function varies depending on whether
a body is provided.
The function `patch/3,4` does not send a body. It must be
sent separately using link:man:gun:data(3)[gun:data(3)].
The function `patch/4,5` sends the entire request, including
the request body, immediately. It is therefore not possible
to use link:man:gun:data(3)[gun:data(3)] after that. You
should provide a content-type header. Gun will set the
content-length header automatically.
The body sent in this request should be a patch document
with instructions on how to update the resource.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
Body::
Request body.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *2.0*: Implicit body detection has been removed. The body
must now be provided either directly (even if empty)
or using separate calls.
* *1.0*: Function introduced.
== Examples
.Patch a resource
[source,erlang]
----
StreamRef = gun:patch(ConnPid, "/users/1",
[{<<"content-type">>, <<"application/json-patch+json">>}],
<<"[{\"op\":\"add\",\"path\":\"/baz\",\"value\":\"qux\"}]">>).
----
.Patch a resource in multiple calls
[source,erlang]
----
StreamRef = gun:patch(ConnPid, "/users/1", [
{<<"content-type">>, <<"application/json-patch+json">>}
]).
gun:data(ConnPid, StreamRef, fin,
<<"[{\"op\":\"add\",\"path\":\"/baz\",\"value\":\"qux\"}]">>).
----
.Patch a resource with request options
[source,erlang]
----
StreamRef = gun:patch(ConnPid, "/users/1",
[{<<"content-type">>, <<"application/json-patch+json">>}],
<<"[{\"op\":\"add\",\"path\":\"/baz\",\"value\":\"qux\"}]">>,
#{reply_to => ReplyToPid}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

120
gun/doc/src/manual/gun.post.asciidoc Normal file
View File

@ -0,0 +1,120 @@
= gun:post(3)
== Name
gun:post - Process the enclosed representation according to a resource's own semantics
== Description
[source,erlang]
----
post(ConnPid, Path, Headers)
-> post(ConnPid, Path, Headers, #{})
post(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
post(ConnPid, Path, Headers, Body)
-> post(ConnPid, Path, Headers, Body, #{})
post(ConnPid, Path, Headers, Body, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
Body :: iodata()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Process the enclosed representation according to a resource's
own semantics.
The behavior of this function varies depending on whether
a body is provided.
The function `post/3,4` does not send a body. It must be
sent separately using link:man:gun:data(3)[gun:data(3)].
The function `post/4,5` sends the entire request, including
the request body, immediately. It is therefore not possible
to use link:man:gun:data(3)[gun:data(3)] after that. You
should provide a content-type header. Gun will set the
content-length header automatically.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
Body::
Request body.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *2.0*: Implicit body detection has been removed. The body
must now be provided either directly (even if empty)
or using separate calls.
* *1.0*: Function introduced.
== Examples
.Post to a resource
[source,erlang]
----
StreamRef = gun:post(ConnPid, "/search",
[{<<"content-type">>, <<"application/x-www-form-urlencoded">>}],
<<"q=nine%20nines">>).
----
.Post to a resource in multiple calls
[source,erlang]
----
StreamRef = gun:post(ConnPid, "/search", [
{<<"content-type">>, <<"application/x-www-form-urlencoded">>}
]).
gun:data(ConnPid, StreamRef, fin, <<"q=nine%20nines">>).
----
.Post to a resource with request options
[source,erlang]
----
StreamRef = gun:post(ConnPid, "/search",
[{<<"content-type">>, <<"application/x-www-form-urlencoded">>}],
<<"q=nine%20nines">>,
#{reply_to => ReplyToPid}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

119
gun/doc/src/manual/gun.put.asciidoc Normal file
View File

@ -0,0 +1,119 @@
= gun:put(3)
== Name
gun:put - Create or replace a resource
== Description
[source,erlang]
----
put(ConnPid, Path, Headers)
-> put(ConnPid, Path, Headers, #{})
put(ConnPid, Path, Headers, ReqOpts)
-> StreamRef
put(ConnPid, Path, Headers, Body)
-> put(ConnPid, Path, Headers, Body, #{})
put(ConnPid, Path, Headers, Body, ReqOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
Body :: iodata()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Create or replace a resource.
The behavior of this function varies depending on whether
a body is provided.
The function `put/3,4` does not send a body. It must be
sent separately using link:man:gun:data(3)[gun:data(3)].
The function `put/4,5` sends the entire request, including
the request body, immediately. It is therefore not possible
to use link:man:gun:data(3)[gun:data(3)] after that. You
should provide a content-type header. Gun will set the
content-length header automatically.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
Body::
Request body.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *2.0*: Implicit body detection has been removed. The body
must now be provided either directly (even if empty)
or using separate calls.
* *1.0*: Function introduced.
== Examples
.Put a resource
[source,erlang]
----
StreamRef = gun:put(ConnPid, "/lang/fr_FR/hello",
[{<<"content-type">>, <<"text/plain">>}],
<<"Bonjour !">>).
----
.Put a resource in multiple calls
[source,erlang]
----
StreamRef = gun:put(ConnPid, "/lang/fr_FR/hello", [
{<<"content-type">>, <<"text/plain">>}
]).
gun:data(ConnPid, StreamRef, fin, <<"Bonjour !">>).
----
.Put a resource with request options
[source,erlang]
----
StreamRef = gun:put(ConnPid, "/lang/fr_FR/hello",
[{<<"content-type">>, <<"text/plain">>}],
<<"Bonjour !">>,
#{reply_to => ReplyToPid}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

97
gun/doc/src/manual/gun.request.asciidoc Normal file
View File

@ -0,0 +1,97 @@
= gun:request(3)
== Name
gun:request - Perform the given request
== Description
[source,erlang]
----
request(ConnPid, Method, Path, Headers, Body)
-> request(ConnPid, Method, Path, Headers, Body, #{})
request(ConnPid, Method, Path, Headers, Body, ReqOpts)
-> StreamRef
ConnPid :: pid()
Method :: binary()
Path :: iodata()
Headers :: gun:req_headers()
Body :: iodata()
ReqOpts :: gun:req_opts()
StreamRef :: gun:stream_ref()
----
Perform the given request.
This is a general purpose function that should only be
used when other method-specific functions do not apply.
The function `request/5,6` sends the entire request, including
the request body, immediately. It is therefore not possible
to use link:man:gun:data(3)[gun:data(3)] after that. You
should provide a content-type header. Gun will set the
content-length header automatically.
== Arguments
ConnPid::
The pid of the Gun connection process.
Method::
Method to be used for the request.
Path::
Path to the resource.
Headers::
Additional request headers.
Body::
Request body.
ReqOpts::
Request options.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *2.0*: Implicit body detection has been removed. The body
must now be provided either directly (even if empty)
or using link:man:gun:headers(3)[gun:headers(3)].
* *1.0*: Function introduced.
== Examples
.Perform a request
[source,erlang]
----
StreamRef = gun:request(ConnPid, <<"PUT">>,
"/lang/fr_FR/hello",
[{<<"content-type">>, <<"text/plain">>}],
<<"Bonjour !">>).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:await(3)[gun:await(3)],
link:man:gun:await_body(3)[gun:await_body(3)],
link:man:gun_push(3)[gun_push(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

57
gun/doc/src/manual/gun.set_owner.asciidoc Normal file
View File

@ -0,0 +1,57 @@
= gun:set_owner(3)
== Name
gun:set_owner - Set a new owner for the connection
== Description
[source,erlang]
----
set_owner(ConnPid, OwnerPid) -> ok
ConnPid :: pid()
OwnerPid :: pid()
----
Set a new owner for the connection.
Only the current owner of the connection can set a new
owner.
Gun monitors the owner of the connection and automatically
shuts down gracefully when the owner exits.
== Arguments
ConnPid::
The pid of the Gun connection process.
OwnerPid::
The pid of the new owner for the connection.
== Return value
The atom `ok` is returned.
== Changelog
* *2.0*: Function introduced.
== Examples
.Set a new owner for the connection
[source,erlang]
----
ok = gun:set_owner(ConnPid, OwnerPid).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:shutdown(3)[gun:shutdown(3)],
link:man:gun:close(3)[gun:close(3)]

67
gun/doc/src/manual/gun.shutdown.asciidoc Normal file
View File

@ -0,0 +1,67 @@
= gun:shutdown(3)
== Name
gun:shutdown - Gracefully close the connection
== Description
[source,erlang]
----
shutdown(ConnPid) -> ok
ConnPid :: pid()
----
Gracefully close the connection.
Gun will wait for up to `closing_timeout` milliseconds
before brutally closing the connection. The graceful
shutdown mechanism varies between the different protocols:
* For HTTP/1.1 there is no such mechanism and Gun will
close the connection once the current response is
received. Any pipelined requests are immediately
terminated.
* For HTTP/2 Gun will send a GOAWAY frame and wait for
the existing streams to terminate.
* For Websocket Gun will send a close frame and wait
for the server's close frame before closing the
connection.
The function returns immediately. The connection may
therefore still be up for some time after this call.
Gun will not attempt to reconnect once graceful
shutdown has been initiated.
== Arguments
ConnPid::
The pid of the Gun connection process.
== Return value
The atom `ok` is returned.
== Changelog
* *2.0*: Function introduced.
== Examples
.Gracefully shutdown the connection
[source,erlang]
----
ok = gun:shutdown(ConnPid).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:close(3)[gun:close(3)]

79
gun/doc/src/manual/gun.stream_info.asciidoc Normal file
View File

@ -0,0 +1,79 @@
= gun:stream_info(3)
== Name
gun:stream_info - Obtain information about a stream
== Description
[source,erlang]
----
stream_info(ConnPid, StreamRef) -> {ok, undefined | Info} | {error, not_connected}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Info :: #{
ref => gun:stream_ref(),
reply_to => pid(),
state => running | stopping,
intermediaries => [Intermediary],
tunnel => Tunnel
}
Intermediary :: #{
type => connect | socks5,
host => inet:hostname() | inet:ip_address(),
port => inet:port_number(),
transport => tcp | tls,
protocol => http | http2 | socks | raw
}
Tunnel :: #{
transport => tcp | tls,
protocol => http | http2 | socks | raw,
origin_scheme => binary() | undefined,
origin_host => inet:hostname() | inet:ip_address(),
origin_port => inet:port_number()
}
----
Obtain information about a stream.
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
== Return value
A map is returned containing various informations about
the stream.
== Changelog
* *2.0*: Function introduced.
== Examples
.Obtain information about a stream
[source,erlang]
----
Info = gun:stream_info(ConnPid, StreamRef).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:cancel(3)[gun:cancel(3)]

67
gun/doc/src/manual/gun.update_flow.asciidoc Normal file
View File

@ -0,0 +1,67 @@
= gun:update_flow(3)
== Name
gun:update_flow - Update a stream's flow control value
== Description
[source,erlang]
----
update_flow(ConnPid, StreamRef, Flow) -> ok
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Flow :: pos_integer()
----
Update a stream's flow control value.
The flow value can only ever be incremented.
This function does nothing for streams that have flow
control disabled (which is the default).
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
Flow::
Flow control value increment.
== Return value
The atom `ok` is returned.
== Changelog
* *2.0*: Function introduced.
== Examples
.Update a stream's flow control value
[source,erlang]
----
gun:update_flow(ConnPid, StreamRef, 10).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)]

72
gun/doc/src/manual/gun.ws_send.asciidoc Normal file
View File

@ -0,0 +1,72 @@
= gun:ws_send(3)
== Name
gun:ws_send - Send Websocket frames
== Description
[source,erlang]
----
ws_send(ConnPid, StreamRef, Frames) -> ok
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Frames :: Frame | [Frame]
Frame :: close | ping | pong
| {text | binary | close | ping | pong, iodata()}
| {close, non_neg_integer(), iodata()}
----
Send Websocket frames.
The connection must first be upgraded to Websocket using
the function link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)].
== Arguments
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream that was upgraded to Websocket.
Frames::
One or more Websocket frame(s).
== Return value
The atom `ok` is returned.
== Changelog
* *2.0*: The mandatory `StreamRef` argument was added.
* *2.0*: It is now possible to send multiple frames at once.
* *1.0*: Function introduced.
== Examples
.Send a single frame
[source,erlang]
----
gun:ws_send(ConnPid, StreamRef, {text, <<"Hello world!">>}).
----
.Send many frames including a close frame
[source,erlang]
----
gun:ws_send(ConnPid, StreamRef, [
{text, <<"See you later, world!">>},
close
]).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)],
link:man:gun_upgrade(3)[gun_upgrade(3)],
link:man:gun_ws(3)[gun_ws(3)]

117
gun/doc/src/manual/gun.ws_upgrade.asciidoc Normal file
View File

@ -0,0 +1,117 @@
= gun:ws_upgrade(3)
== Name
gun:ws_upgrade - Upgrade to Websocket
== Description
[source,erlang]
----
ws_upgrade(ConnPid, Path)
-> ws_upgrade(ConnPid, Path, [])
ws_upgrade(ConnPid, Path, Headers)
-> StreamRef
ws_upgrade(ConnPid, Path, Headers, WsOpts)
-> StreamRef
ConnPid :: pid()
Path :: iodata()
Headers :: gun:req_headers()
WsOpts :: gun:ws_opts()
StreamRef :: gun:stream_ref()
----
Upgrade to Websocket.
The behavior of this function depends on the protocol
selected.
HTTP/1.1 cannot handle Websocket and HTTP requests
concurrently. The upgrade, if successful, will result
in the complete takeover of the connection. Any
subsequent HTTP requests will be rejected.
Gun does not currently support Websocket over HTTP/2.
By default Gun will take the Websocket options from
the connection's `ws_opts`.
Websocket subprotocol negotiation is enabled when
the `protocols` option is given. It takes a subprotocol
name and a module implementing the
link:man:gun_ws_protocol(3)[gun_ws_protocol(3)] behavior.
== Arguments
ConnPid::
The pid of the Gun connection process.
Path::
Path to the resource.
Headers::
Additional request headers.
WsOpts::
Configuration for the Websocket protocol.
== Return value
A reference that identifies the newly created stream is
returned. It is this reference that must be passed in
subsequent calls and will be received in messages related
to this new stream.
== Changelog
* *1.0*: Function introduced.
== Examples
.Upgrade to Websocket
[source,erlang]
----
StreamRef = gun:ws_upgrade(ConnPid, "/ws", [
{<<"sec-websocket-protocol">>, <<"chat">>}
]).
receive
{gun_upgrade, ConnPid, StreamRef, [<<"websocket">>], _} ->
ok
after 5000 ->
error(timeout)
end.
----
.Upgrade to Websocket with different options
[source,erlang]
----
StreamRef = gun:ws_upgrade(ConnPid, "/ws", [], #{
compress => false
}).
----
.Upgrade to Websocket with protocol negotiation
[source,erlang]
----
StreamRef = gun:ws_upgrade(ConnPid, "/ws", [], #{
protocols => [
{<<"mqtt">>, gun_ws_mqtt_h},
{<<"v12.stomp">>, gun_ws_stomp_h}
]
}).
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:ws_send(3)[gun:ws_send(3)],
link:man:gun_upgrade(3)[gun_upgrade(3)],
link:man:gun_ws(3)[gun_ws(3)],
link:man:gun_ws_protocol(3)[gun_ws_protocol(3)]

44
gun/doc/src/manual/gun_app.asciidoc Normal file
View File

@ -0,0 +1,44 @@
= gun(7)
== Name
gun - HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP
== Description
Gun is an HTTP client for Erlang/OTP with support for the
HTTP/1.1, HTTP/2 and Websocket protocols.
Gun aims to provide an easy to use, asynchronous and
always-connected client. It maintains a permanent connection
to the server and reconnects automatically when necessary.
== Modules
* link:man:gun(3)[gun(3)] - Asynchronous HTTP client
* link:man:gun_cookies(3)[gun_cookies(3)] - Cookie store engine
* link:man:gun_cookies_list(3)[gun_cookies_list(3)] - Cookie store backend: in-memory, per connection
* link:man:gun_event(3)[gun_event(3)] - Events
* link:man:gun_ws_protocol(3)[gun_ws_protocol(3)] - Websocket subprotocols
== Dependencies
* link:man:cowlib(7)[cowlib(7)] - Support library for manipulating Web protocols
* ssl - Secure communication over sockets
All these applications must be started before the `gun`
application. To start Gun and all dependencies at once:
[source,erlang]
----
{ok, _} = application:ensure_all_started(gun).
----
== Environment
The `gun` application does not define any application
environment configuration parameters.
== See also
link:man:cowlib(7)[cowlib(7)]

179
gun/doc/src/manual/gun_cookies.asciidoc Normal file
View File

@ -0,0 +1,179 @@
= gun_cookies(3)
== Name
gun_cookies - Cookie store engine
== Description
The `gun_cookies` module implements a cookie store engine.
It will be used by Gun when a cookie store is configured.
It also defines the interface and provides functions used
to implement cookie store backends.
== Callbacks
Cookie store backends implement the following interface.
Functions are organized by theme: initialization, querying,
storing and garbage collecting:
=== init
[source,erlang]
----
init(Opts :: any()) -> gun_cookies:store()
----
Initialize the cookie store.
=== query
[source,erlang]
----
query(State, URI) -> {ok, [Cookie], State}
URI :: uri_string:uri_map()
Cookie :: gun_cookies:cookie()
State :: any()
----
Query the store for the cookies for the given URI.
=== set_cookie_secure_match
[source,erlang]
----
set_cookie_secure_match(State, Match) -> match | nomatch
State :: any()
Match :: #{
name := binary(),
% secure_only := true,
domain := binary(),
path := binary()
}
----
Perform a secure match against cookies already in the store.
This is part of the heuristics that the cookie store engine
applies to decide whether the cookie must be stored.
The `secure_only` attribute is implied, it is not actually
passed in the argument.
=== set_cookie_get_exact_match
[source,erlang]
----
set_cookie_get_exact_match(State, Match)
-> {ok, gun_cookies:cookie(), State} | error
State :: any()
Match :: #{
name := binary(),
domain := binary(),
host_only := boolean(),
path := binary()
}
----
Perform an exact match against cookies already in the store.
This is part of the heuristics that the cookie store engine
applies to decide whether the cookie must be stored.
When a cookie is found, it must be returned so that it gets
updated. When nothing is found a new cookie will be stored.
=== store
[source,erlang]
----
store(State, gun_cookies:cookie())
-> {ok, State} | {error, any()}
State :: any()
----
Unconditionally store the cookie into the cookie store.
=== gc
[source,erlang]
----
gc(State) -> {ok, State}
State :: any()
----
Remove all cookies from the cookie store that are expired.
Other cookies may be removed as well, at the discretion
of the cookie store. For example excess cookies may be
removed to reduce the memory footprint.
=== session_gc
[source,erlang]
----
session_gc(State) -> {ok, State}
State :: any()
----
Remove all cookies from the cookie store that have the
`persistent` flag set to `false`.
== Exports
* link:man:gun_cookies:domain_match(3)[gun_cookies:domain_match(3)] - Cookie domain match
* link:man:gun_cookies:path_match(3)[gun_cookies:path_match(3)] - Cookie path match
== Types
=== cookie()
[source,erlang]
----
cookie() :: #{
name := binary(),
value := binary(),
domain := binary(),
path := binary(),
creation_time := calendar:datetime(),
last_access_time := calendar:datetime(),
expiry_time := calendar:datetime() | infinity,
persistent := boolean(),
host_only := boolean(),
secure_only := boolean(),
http_only := boolean(),
same_site := strict | lax | none
}
----
A cookie.
This contains the cookie name, value, attributes and flags.
This is the representation that the cookie store engine
and Gun expects. Cookies do not have to be kept in this
format in the cookie store backend.
=== store()
[source,erlang]
----
store() :: {module(), StoreState :: any()}
----
The cookie store.
This is a tuple containing the cookie store backend module
and its current state.
== Changelog
* *2.0*: Module introduced.
== See also
link:man:gun(7)[gun(7)],
link:man:gun_cookies_list(3)[gun_cookies_list(3)]

52
gun/doc/src/manual/gun_cookies.domain_match.asciidoc Normal file
View File

@ -0,0 +1,52 @@
= gun_cookies:domain_match(3)
== Name
gun_cookies:domain_match - Cookie domain match
== Description
[source,erlang]
----
domain_match(String, DomainString) -> boolean()
String :: binary()
DomainString :: binary()
----
Cookie domain match.
This function can be used when implementing the
`set_cookie_secure_match` callback of a cookie store.
== Arguments
String::
The string to match.
DomainString::
The domain string that will be matched against.
== Return value
Returns `true` when `String` domain-matches `DomainString`,
and `false` otherwise.
== Changelog
* *2.0*: Function introduced.
== Examples
.Perform a domain match
[source,erlang]
----
Match = gun_cookies:domain_match(Domain, CookieDomain).
----
== See also
link:man:gun_cookies(3)[gun_cookies(3)],
link:man:gun_cookies:path_match(3)[gun_cookies:path_match(3)]

52
gun/doc/src/manual/gun_cookies.path_match.asciidoc Normal file
View File

@ -0,0 +1,52 @@
= gun_cookies:path_match(3)
== Name
gun_cookies:path_match - Cookie path match
== Description
[source,erlang]
----
path_match(ReqPath, CookiePath) -> boolean()
ReqPath :: binary()
CookiePath :: binary()
----
Cookie path match.
This function can be used when implementing the
`set_cookie_secure_match` callback of a cookie store.
== Arguments
ReqPath::
The request path to match.
CookiePath::
The cookie path that will be matched against.
== Return value
Returns `true` when `ReqPath` path-matches `CookiePath`,
and `false` otherwise.
== Changelog
* *2.0*: Function introduced.
== Examples
.Perform a path match
[source,erlang]
----
Match = gun_cookies:path_match(ReqPath, CookiePath).
----
== See also
link:man:gun_cookies(3)[gun_cookies(3)],
link:man:gun_cookies:domain_match(3)[gun_cookies:domain_match(3)]

55
gun/doc/src/manual/gun_cookies_list.asciidoc Normal file
View File

@ -0,0 +1,55 @@
= gun_cookies_list(3)
== Name
gun_cookies_list - Cookie store backend: in-memory, per connection
== Description
The `gun_cookies_list` module implements a cookie store
backend that keeps all the cookie data in-memory and tied
to a specific connection.
It is possible to implement a custom backend on top of
`gun_cookies_list` in order to add persistence or sharing
properties.
== Exports
This module implements the callbacks defined in
link:man:gun_cookies(3)[gun_cookies(3)].
== Types
=== opts()
[source,erlang]
----
opts() :: #{
}
----
Cookie store backend options.
There are currently no options available for this backend.
// The default value is given next to the option name:
== Changelog
* *2.0*: Module introduced.
== Examples
.Open a connection with a cookie store configured
[source,erlang]
----
{ok, ConnPid} = gun:open(Host, Port, #{
cookie_store => gun_cookies_list:init(#{})
})
----
== See also
link:man:gun(7)[gun(7)],
link:man:gun_cookies(3)[gun_cookies(3)]

78
gun/doc/src/manual/gun_data.asciidoc Normal file
View File

@ -0,0 +1,78 @@
= gun_data(3)
== Name
gun_data - Response body
== Description
[source,erlang]
----
{gun_data, ConnPid, StreamRef, IsFin, Data}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
IsFin :: fin | nofin
Data :: binary()
----
Response body.
This message informs the relevant process that the server
sent a all or part of the body for the response to the
original request.
A data message is always preceded by a response message.
The response body may be terminated either by a data
message with the flag `fin` set or by a
link:man:gun_trailers(3)[gun_trailers(3)] message.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
IsFin::
Whether this message terminates the response.
Data::
All or part of the response body.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_data message in a gen_server
[source,erlang]
----
handle_info({gun_data, ConnPid, _StreamRef,
_IsFin, _Data},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_trailers(3)[gun_trailers(3)]

79
gun/doc/src/manual/gun_down.asciidoc Normal file
View File

@ -0,0 +1,79 @@
= gun_down(3)
== Name
gun_down - The connection is down
== Description
[source,erlang]
----
{gun_down, ConnPid, Protocol, Reason, KilledStreams}
ConnPid :: pid()
Protocol :: http | http2 | socks | ws
Reason :: any()
KilledStreams :: [gun:stream_ref()]
----
The connection is down.
This message informs the owner process that the connection
was lost. Depending on the `retry` and `retry_timeout`
options Gun may automatically attempt to reconnect.
When the connection goes back up, Gun will not attempt to retry
requests. It will also not upgrade to Websocket automatically
if that was the protocol in use when the connection was lost.
== Elements
ConnPid::
The pid of the Gun connection process.
Protocol::
The protocol that was selected for this connection
or upgraded to during the course of the connection.
Reason::
The reason for the loss of the connection.
+
It is present for debugging purposes only. You should not
rely on this value to perform operations programmatically.
KilledStreams::
List of streams that have been brutally terminated.
+
They are active streams that did not complete before the closing
of the connection. Whether they can be retried safely depends
on the protocol used and the idempotence property of the requests.
== Changelog
* *2.0*: The last element of the message's tuple, `UnprocessedStreams`
has been removed.
* *1.0*: Message introduced.
== Examples
.Receive a gun_down message in a gen_server
[source,erlang]
----
handle_info({gun_down, ConnPid, _Protocol, _Reason, _Killed},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun_up(3)[gun_up(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_error(3)[gun_error(3)]

68
gun/doc/src/manual/gun_error.asciidoc Normal file
View File

@ -0,0 +1,68 @@
= gun_error(3)
== Name
gun_error - Stream or connection-wide error
== Description
[source,erlang]
----
{gun_error, ConnPid, StreamRef, Reason}
{gun_error, ConnPid, Reason}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Reason :: any()
----
Stream or connection-wide error.
These messages inform the relevant process that an error
occurred. A reference is given when the error pertains
to a specific stream. Connection-wide errors do not
imply that the connection is no longer usable, they are
used for all errors that are not specific to a stream.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream that resulted in an error.
Reason::
The reason for the error.
+
It is present for debugging purposes only. You should not
rely on this value to perform operations programmatically.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_error message in a gen_server
[source,erlang]
----
handle_info({gun_error, ConnPid, _Reason},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State};
handle_info({gun_error, ConnPid, _StreamRef, _Reason},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun_up(3)[gun_up(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_down(3)[gun_down(3)]

509
gun/doc/src/manual/gun_event.asciidoc Normal file
View File

@ -0,0 +1,509 @@
= gun_event(3)
== Name
gun_event - Events
== Description
The `gun_event` module provides the callback interface
and types for implementing event handlers.
== Callbacks
Event handlers implement the following interface.
Because types are tied to specific events, they
are documented alongside them. All event types
are exported and can be referred to as `gun_event:Type()`.
The events are ordered by the order they are likely
to be triggered, with the most frequent events listed
first.
=== init
[source,erlang]
----
init_event() :: #{
owner := pid(),
transport := tcp | tls,
origin_scheme := binary(),
origin_host := inet:hostname() | inet:ip_address(),
origin_port := inet:port_number(),
opts := gun:opts()
}
init(init_event(), State) -> State
----
Gun is initializing.
=== domain_lookup_start
[source,erlang]
----
domain_lookup_event() :: #{
host := inet:hostname() | inet:ip_address(),
port := inet:port_number(),
tcp_opts := [gen_tcp:connect_option()],
timeout := timeout(),
lookup_info => gun_tcp:lookup_info(),
error => any()
}
domain_lookup_start(domain_lookup_event(), State) -> State
----
Gun is starting to resolve the host address.
The `lookup_info` and `error` keys are never set
for this event.
=== domain_lookup_end
[source,erlang]
----
domain_lookup_end(domain_lookup_event(), State) -> State
----
Gun has finished resolving the host address.
The `lookup_info` key is only set when the lookup is
successful. The `error` key is set otherwise.
=== connect_start
[source,erlang]
----
connect_event() :: #{
lookup_info := gun_tcp:lookup_info(),
timeout := timeout(),
socket => inet:socket(),
protocol => http | http2 | socks | raw,
error => any()
}
connect_start(connect_event(), State) -> State
----
Gun is starting to connect to the host address and port.
The `socket`, `protocol` and `error` keys are never set
for this event.
=== connect_end
[source,erlang]
----
connect_end(connect_event(), State) -> State
----
Gun has finished connecting to the host address and port.
The `socket` key is set on connect success. The `error`
key is set otherwise.
The `protocol` key is only set when the transport is
`tcp` and the connection is successful. The protocol
is only known in the `tls_handshake_end` event otherwise.
=== tls_handshake_start
[source,erlang]
----
tls_handshake_event() :: #{
stream_ref => gun:stream_ref(),
reply_to => pid(),
socket := inet:socket() | ssl:sslsocket() | pid(), %% The socket before/after will be different.
tls_opts := [ssl:tls_client_option()],
timeout := timeout(),
protocol => http | http2 | socks | raw,
error => any()
}
tls_handshake_start(tls_handshake_event(), State) -> State
----
Gun has started a TLS handshake.
This and the `tls_handshake_end` event only occur when
connecting to a TLS server or when upgrading the connection
or a stream to use TLS, for example when using CONNECT or
when connecting to a secure SOCKS server.
The `stream_ref` and `reply_to` keys are only set when the
TLS handshake occurs as a result of a CONNECT request or
inside an existing CONNECT tunnel.
The `protocol` and `error` keys are never set for this event.
=== tls_handshake_end
[source,erlang]
----
tls_handshake_end(tls_handshake_event(), State) -> State
----
Gun has finished a TLS handshake.
The `protocol` key is set on TLS handshake success. The
`error` key is set otherwise.
=== request_start
[source,erlang]
----
request_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
function := headers | request | ws_upgrade | connect,
method := iodata(),
scheme => binary(),
authority := iodata(),
path => iodata(),
headers := [{binary(), iodata()}]
}
request_start(request_start_event(), State) -> State
----
Gun is starting to send a request.
The `scheme` and `path` keys are never set when the `function`
is set to `connect`.
=== request_headers
[source,erlang]
----
request_headers(request_start_event(), State) -> State
----
Gun has finished sending the request headers.
=== request_end
[source,erlang]
----
request_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}
request_end(request_end_event(), State) -> State
----
Gun has finished sending the request.
=== push_promise_start
[source,erlang]
----
push_promise_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}
push_promise_start(push_promise_start_event(), State) -> State
----
Gun has begun receiving a promised request (server push).
=== push_promise_end
[source,erlang]
----
push_promise_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
promised_stream_ref => gun:stream_ref(),
method := binary(),
uri := binary(),
headers := [{binary(), iodata()}]
}
push_promise_end(push_promise_end_event(), State) -> State
----
Gun has finished receiving a promised request (server push).
Promised requests never include a body.
Promised requests received during the graceful shutdown of the
connection get canceled immediately.
// @todo The cancel event should probably trigger as well.
=== response_start
[source,erlang]
----
response_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}
response_start(response_start_event(), State) -> State
----
Gun has begun receiving a response.
=== response_inform
[source,erlang]
----
response_headers_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
status := non_neg_integer(),
headers := [{binary(), binary()}]
}
response_inform(response_headers_event(), State) -> State
----
Gun has received an informational response (1xx status code).
A `status` with value 101 indicates that the response has
concluded as the stream will be upgraded to a new protocol.
=== response_headers
[source,erlang]
----
response_headers(response_headers_event(), State) -> State
----
Gun has finished receiving response headers.
=== response_trailers
[source,erlang]
----
response_trailers_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
headers := [{binary(), binary()}]
}
response_trailers(response_trailers_event(), State) -> State
----
Gun has received response trailers.
=== response_end
[source,erlang]
----
response_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}
response_end(response_end_event(), State) -> State
----
Gun has finished receiving a response.
=== ws_upgrade
[source,erlang]
----
ws_upgrade_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
opts := gun:ws_opts()
}
ws_upgrade(ws_upgrade_event(), State) -> State
----
A Websocket upgrade was requested.
Success is indicated by a response (101 informational
if HTTP/1.1, 2xx if HTTP/2) followed by a `protocol_changed`
event.
=== ws_recv_frame_start
[source,erlang]
----
ws_recv_frame_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
frag_state := cow_ws:frag_state(),
extensions := cow_ws:extensions()
}
ws_recv_frame_start(ws_recv_frame_start_event(), State) -> State
----
Gun has begun receiving a Websocket frame.
=== ws_recv_frame_header
[source,erlang]
----
ws_recv_frame_header_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
frag_state := cow_ws:frag_state(),
extensions := cow_ws:extensions(),
type := cow_ws:frame_type(),
rsv := cow_ws:rsv(),
len := non_neg_integer(),
mask_key := cow_ws:mask_key()
}
ws_recv_frame_header(ws_recv_frame_header_event(), State) -> State
----
Gun has received the header part of a Websocket frame.
It will be immediately be followed by the frame's payload.
=== ws_recv_frame_end
[source,erlang]
----
ws_recv_frame_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
extensions := cow_ws:extensions(),
close_code := undefined | cow_ws:close_code(),
payload := binary()
}
ws_recv_frame_end(ws_recv_frame_end_event(), State) -> State
----
Gun has finished receiving a Websocket frame.
=== ws_send_frame_start
[source,erlang]
----
ws_send_frame_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
extensions := cow_ws:extensions(),
frame := gun:ws_frame()
}
ws_send_frame_start(ws_send_frame_event(), State) -> State
----
Gun has started sending a Websocket frame.
=== ws_send_frame_end
[source,erlang]
----
ws_send_frame_end(ws_send_frame_event(), State) -> State
----
Gun has finished sending a Websocket frame.
=== protocol_changed
[source,erlang]
----
protocol_changed_event() :: #{
stream_ref := gun:stream_ref(),
protocol := http | http2 | socks | raw | ws
}
protocol_changed(protocol_changed_event(), State) -> State
----
The protocol has changed for either the entire Gun connection
or for a specific stream.
The `stream_ref` key is only set when the protocol has
changed for a specific stream or inside a CONNECT tunnel.
This event occurs during successful Websocket upgrades,
as a result of successful CONNECT requests or after a
SOCKS tunnel was successfully established.
=== origin_changed
[source,erlang]
----
origin_changed_event() :: #{
stream_ref => gun:stream_ref(),
type := connect | socks5,
origin_scheme := binary(),
origin_host := inet:hostname() | inet:ip_address(),
origin_port := inet:port_number()
}
origin_changed(origin_changed_event(), State) -> State
----
The origin server has changed for either the Gun connection
or for a specific stream.
The `stream_ref` key is only set when the origin has
changed for a specific stream or inside a CONNECT tunnel.
=== cancel
[source,erlang]
----
cancel_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
endpoint := local | remote,
reason := atom()
}
cancel(cancel_event(), State) -> State
----
A stream has been canceled.
HTTP/1.1 streams can't be canceled at the protocol level. In
this case Gun will silence the stream for the user but events
may still occur.
HTTP/2 streams can be canceled both by the client and the
server. Events may still occur for a short time after the
stream has been canceled.
=== disconnect
[source,erlang]
----
disconnect_event() :: #{
reason := normal | closed | {error, any()}
}
disconnect(disconnect_event(), State) -> State
----
Gun has been disconnected from the server.
=== terminate
[source,erlang]
----
terminate_event() :: #{
state := not_connected
| domain_lookup | connecting | initial_tls_handshake | tls_handshake
| connected | connected_data_only | connected_ws_only,
reason := normal | shutdown | {shutdown, any()} | any()
}
terminate(terminate_event(), State) -> State
----
Gun is terminating.
== Changelog
* *2.0*: Module introduced.
== See also
link:man:gun(7)[gun(7)],
link:man:gun(3)[gun(3)]

70
gun/doc/src/manual/gun_inform.asciidoc Normal file
View File

@ -0,0 +1,70 @@
= gun_inform(3)
== Name
gun_inform - Informational response
== Description
[source,erlang]
----
{gun_inform, ConnPid, StreamRef, Status, Headers}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Status :: 100..199
Headers :: [{binary(), binary()}]
----
Informational response.
This message informs the relevant process that the server
sent an informational response to the original request.
Informational responses are only intermediate responses
and provide no guarantees as to what the final response
will be. An informational response always precedes the
response to the original request.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
Status::
Status code for the informational response.
Headers::
Headers sent with the informational response.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_inform message in a gen_server
[source,erlang]
----
handle_info({gun_inform, ConnPid, _StreamRef,
_Status, _Headers},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun_response(3)[gun_response(3)]

90
gun/doc/src/manual/gun_push.asciidoc Normal file
View File

@ -0,0 +1,90 @@
= gun_push(3)
== Name
gun_push - Server-initiated push
== Description
[source,erlang]
----
{gun_push, ConnPid, StreamRef, NewStreamRef, Method, URI, Headers}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
NewStreamRef :: gun:stream_ref()
Method :: binary()
URI :: binary()
Headers :: [{binary(), binary()}]
----
Server-initiated push.
This message informs the relevant process that the server
is pushing a resource related to the effective target URI
of the original request.
A server-initiated push message always precedes the response
to the original request.
This message will not be sent when using the HTTP/1.1 protocol
because it lacks the concept of server-initiated push.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
NewStreamRef::
Identifier of the stream being pushed.
Method::
Method of the equivalent HTTP request.
URI::
URI of the resource being pushed.
Headers::
Headers of the equivalent HTTP request.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_push message in a gen_server
[source,erlang]
----
handle_info({gun_push, ConnPid, _StreamRef,
_NewStreamRef, _Method, _URI, _Headers},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
.Cancel an unwanted push
[source,erlang]
----
handle_info({gun_push, ConnPid, _StreamRef,
NewStreamRef, _Method, _URI, _Headers},
State=#state{conn_pid=ConnPid}) ->
gun:cancel(ConnPid, NewStreamRef),
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:cancel(3)[gun:cancel(3)],
link:man:gun_response(3)[gun_response(3)]

76
gun/doc/src/manual/gun_response.asciidoc Normal file
View File

@ -0,0 +1,76 @@
= gun_response(3)
== Name
gun_response - Response
== Description
[source,erlang]
----
{gun_response, ConnPid, StreamRef, IsFin, Status, Headers}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
IsFin :: fin | nofin
Status :: non_neg_integer()
Headers :: [{binary(), binary()}]
----
Response.
This message informs the relevant process that the server
sent a response to the original request.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
IsFin::
Whether this message terminates the response.
Status::
Status code for the response.
Headers::
Headers sent with the response.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_response message in a gen_server
[source,erlang]
----
handle_info({gun_response, ConnPid, _StreamRef,
_IsFin, _Status, _Headers},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun_inform(3)[gun_inform(3)],
link:man:gun_push(3)[gun_push(3)]

68
gun/doc/src/manual/gun_trailers.asciidoc Normal file
View File

@ -0,0 +1,68 @@
= gun_trailers(3)
== Name
gun_trailers - Response trailers
== Description
[source,erlang]
----
{gun_trailers, ConnPid, StreamRef, Headers}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Headers :: [{binary(), binary()}]
----
Response trailers.
This message informs the relevant process that the server
sent response trailers for the response to the original
request.
A trailers message terminates the response.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream for the original request.
Headers::
Trailing headers sent after the response body.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_trailers message in a gen_server
[source,erlang]
----
handle_info({gun_trailers, ConnPid, _StreamRef, _Headers},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:get(3)[gun:get(3)],
link:man:gun:head(3)[gun:head(3)],
link:man:gun:patch(3)[gun:patch(3)],
link:man:gun:post(3)[gun:post(3)],
link:man:gun:put(3)[gun:put(3)],
link:man:gun:delete(3)[gun:delete(3)],
link:man:gun:options(3)[gun:options(3)],
link:man:gun:headers(3)[gun:headers(3)],
link:man:gun:request(3)[gun:request(3)],
link:man:gun_response(3)[gun_response(3)],
link:man:gun_data(3)[gun_data(3)]

66
gun/doc/src/manual/gun_tunnel_up.asciidoc Normal file
View File

@ -0,0 +1,66 @@
= gun_tunnel_up(3)
== Name
gun_tunnel_up - The tunnel is up
== Description
[source,erlang]
----
{gun_tunnel_up, ConnPid, StreamRef, Protocol}
ConnPid :: pid()
StreamRef :: gun:stream_ref() | undefined
Protocol :: http | http2 | socks
----
The tunnel is up.
This message informs the owner/calling process that the connection
completed through the SOCKS or CONNECT proxy.
If Gun is configured to connect to another SOCKS server, then the
connection is not usable yet. One or more
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)] messages will follow.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
The stream reference the tunnel is running on, or `undefined`
if there are no underlying stream.
Protocol::
The protocol selected for this connection. It can be used
to determine the capabilities of the server.
== Changelog
* *2.0*: Message introduced.
== Examples
.Receive a gun_tunnel_up message in a gen_server
[source,erlang]
----
handle_info({gun_tunnel_up, ConnPid, _StreamRef, _Protocol},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:await_up(3)[gun:await_up(3)],
link:man:gun_up(3)[gun_up(3)],
link:man:gun_down(3)[gun_down(3)],
link:man:gun_error(3)[gun_error(3)]

66
gun/doc/src/manual/gun_up.asciidoc Normal file
View File

@ -0,0 +1,66 @@
= gun_up(3)
== Name
gun_up - The connection is up
== Description
[source,erlang]
----
{gun_up, ConnPid, Protocol}
ConnPid :: pid()
Protocol :: http | http2 | raw | socks
----
The connection is up.
This message informs the owner process that the connection or
reconnection completed.
If Gun is configured to connect to a Socks server, then the
connection is not usable yet. One or more
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)] messages will follow.
Otherwise, Gun will start processing the messages it received while
waiting for the connection to be up. If this is a reconnection,
then this may not be desirable for all requests. Those requests
should be cancelled when the connection goes down, and any
subsequent messages ignored.
== Elements
ConnPid::
The pid of the Gun connection process.
Protocol::
The protocol selected for this connection. It can be used
to determine the capabilities of the server.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_up message in a gen_server
[source,erlang]
----
handle_info({gun_up, ConnPid, _Protocol},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:open(3)[gun:open(3)],
link:man:gun:open_unix(3)[gun:open_unix(3)],
link:man:gun:await_up(3)[gun:await_up(3)],
link:man:gun_tunnel_up(3)[gun_tunnel_up(3)],
link:man:gun_down(3)[gun_down(3)],
link:man:gun_error(3)[gun_error(3)]

72
gun/doc/src/manual/gun_upgrade.asciidoc Normal file
View File

@ -0,0 +1,72 @@
= gun_upgrade(3)
== Name
gun_upgrade - Successful protocol upgrade
== Description
[source,erlang]
----
{gun_upgrade, ConnPid, StreamRef, Protocols, Headers}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Protocols :: [<<"websocket">>]
Headers :: [{binary(), binary()}]
----
Successful protocol upgrade.
This message informs the relevant process that the server
accepted to upgrade to one or more protocols given in the
original request.
The exact semantics of this message depend on the original
protocol. HTTP/1.1 upgrades apply to the entire connection.
HTTP/2 uses a different mechanism which allows switching
specific streams to a different protocol.
Gun currently only supports upgrading HTTP/1.1 connections
to the Websocket protocol.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream that resulted in an upgrade.
Protocols::
List of protocols this stream was upgraded to.
Headers::
Headers sent with the upgrade response.
== Changelog
* *1.0*: Message introduced.
== Examples
.Receive a gun_upgrade message in a gen_server
[source,erlang]
----
handle_info({gun_upgrade, ConnPid, _StreamRef,
_Protocols, _Headers},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)],
link:man:gun:ws_send(3)[gun:ws_send(3)],
link:man:gun_ws(3)[gun_ws(3)]

65
gun/doc/src/manual/gun_ws.asciidoc Normal file
View File

@ -0,0 +1,65 @@
= gun_ws(3)
== Name
gun_ws - Websocket frame
== Description
[source,erlang]
----
{gun_ws, ConnPid, StreamRef, Frame}
ConnPid :: pid()
StreamRef :: gun:stream_ref()
Frame :: close | ping | pong
| {text | binary | close, binary()}
| {close, non_neg_integer(), binary()}
| {ping | pong, binary()}
----
Websocket frame.
This message informs the relevant process that the server
sent the enclosed frame.
This message can only be sent on streams that were upgraded
to the Websocket protocol.
== Elements
ConnPid::
The pid of the Gun connection process.
StreamRef::
Identifier of the stream that was upgraded to Websocket.
Frame::
The Websocket frame in question.
== Changelog
* *2.0*: Depending on the option `silence_pings`, ping and
pong frames may be sent as well.
* *1.0*: Message introduced.
== Examples
.Receive a gun_ws message in a gen_server
[source,erlang]
----
handle_info({gun_ws, ConnPid, _StreamRef, _Frame},
State=#state{conn_pid=ConnPid}) ->
%% Do something.
{noreply, State}.
----
== See also
link:man:gun(3)[gun(3)],
link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)],
link:man:gun:ws_send(3)[gun:ws_send(3)],
link:man:gun_upgrade(3)[gun_upgrade(3)]

108
gun/doc/src/manual/gun_ws_protocol.asciidoc Normal file
View File

@ -0,0 +1,108 @@
= gun_ws_protocol(3)
== Name
gun_ws_protocol - Websocket subprotocols
== Description
The `gun_ws_protocol` module provides the callback interface
and types for implementing Websocket subprotocols.
== Callbacks
Websocket subprotocols implement the following interface.
=== init
[source,erlang]
----
init(ReplyTo, StreamRef, Headers, Opts) -> {ok, State}
ReplyTo :: pid()
StreamRef :: reference()
Headers :: cow_http:headers()
Opts :: gun:ws_opts()
State :: protocol_state()
----
Initialize the Websocket protocol.
ReplyTo::
The pid of the process that owns the stream and to
which messages will be sent to.
StreamRef::
The reference for the stream. Must be sent in messages
to distinguish between different streams.
Headers::
Headers that were sent in the response establishing
the Websocket connection.
Opts::
Websocket options. Custom options can be provided in
the `user_opts` key.
State::
State for the protocol.
=== handle
[source,erlang]
----
handle(Frame, State) -> {ok, FlowDec, State}
Frame :: cow_ws:frame()
State :: protocol_state()
FlowDec :: non_neg_integer()
----
Handle a Websocket frame.
This callback may receive fragmented frames depending
on the protocol and may need to rebuild the full
frame to process it.
Frame::
Websocket frame.
State::
State for the protocol.
FlowDec::
How many messages were sent. Used to update the flow
control state when the feature is enabled.
== Types
=== protocol_state()
[source,erlang]
----
protocol_state() :: any()
----
State for the protocol.
As this part of the implementation of the protocol
the type may differ between different Websocket
protocol modules.
== Changelog
* *2.0*: Module introduced.
== See also
link:man:gun(7)[gun(7)],
link:man:gun(3)[gun(3)],
link:man:gun:ws_upgrade(3)[gun:ws_upgrade(3)]

10
gun/ebin/gun.app Normal file
View File

@ -0,0 +1,10 @@
{application, 'gun', [
{description, "HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP."},
{vsn, "2.1.0"},
{modules, ['gun','gun_app','gun_conns_sup','gun_content_handler','gun_cookies','gun_cookies_list','gun_data_h','gun_default_event_h','gun_event','gun_http','gun_http2','gun_pool','gun_pool_events_h','gun_pools_sup','gun_protocols','gun_public_suffix','gun_raw','gun_socks','gun_sse_h','gun_sup','gun_tcp','gun_tcp_proxy','gun_tls','gun_tls_proxy','gun_tls_proxy_cb','gun_tls_proxy_http2_connect','gun_tunnel','gun_ws','gun_ws_h','gun_ws_protocol']},
{registered, [gun_sup]},
{applications, [kernel,stdlib,public_key,ssl,cowlib]},
{optional_applications, []},
{mod, {gun_app, []}},
{env, []}
]}.

23
gun/src/Makefile Normal file
View File

@ -0,0 +1,23 @@
.PHONY: all
.SUFFIXES: .erl .beam
ERLC?= erlc -server
OBJS= gun_ws_protocol.beam gun.beam gun_app.beam gun_conns_sup.beam
OBJS+= gun_cookies.beam gun_cookies_list.beam gun_data_h.beam
OBJS+= gun_default_event_h.beam gun_event.beam gun_http.beam
OBJS+= gun_http2.beam gun_pool.beam gun_pool_events_h.beam gun_pools_sup.beam
OBJS+= gun_protocols.beam gun_public_suffix.beam gun_raw.beam
OBJS+= gun_socks.beam gun_sse_h.beam gun_sup.beam gun_tcp.beam
OBJS+= gun_tcp_proxy.beam gun_tls.beam gun_tls_proxy.beam
OBJS+= gun_tls_proxy_cb.beam gun_tls_proxy_http2_connect.beam
OBJS+= gun_tunnel.beam gun_ws.beam gun_ws_h.beam gun_content_handler.beam
all: ${OBJS}
.erl.beam:
${ERLC} ${ERLOPTS} ${ERLFLAGS} $<
clean:
rm -f *.beam

1868
gun/src/gun.erl Normal file
View File

File diff suppressed because it is too large Load Diff

30
gun/src/gun_app.erl Normal file
View File

@ -0,0 +1,30 @@
%% Copyright (c) 2013-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
%% @private
-module(gun_app).
-behaviour(application).
%% API.
-export([start/2]).
-export([stop/1]).
%% API.
start(_Type, _Args) ->
gun_pools = ets:new(gun_pools, [ordered_set, public, named_table]),
gun_sup:start_link().
stop(_State) ->
ok.

36
gun/src/gun_conns_sup.erl Normal file
View File

@ -0,0 +1,36 @@
%% Copyright (c) 2013-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_conns_sup).
-behaviour(supervisor).
%% API.
-export([start_link/0]).
%% supervisor.
-export([init/1]).
%% API.
-spec start_link() -> {ok, pid()}.
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
%% supervisor.
init([]) ->
Procs = [
#{id => gun, start => {gun, start_link, []}, restart => temporary}
],
{ok, {#{strategy => simple_one_for_one}, Procs}}.

76
gun/src/gun_content_handler.erl Normal file
View File

@ -0,0 +1,76 @@
%% Copyright (c) 2017-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_content_handler).
-export([init/5]).
-export([handle/3]).
-export([check_option/1]).
-type opt() :: [module() | {module(), map()}].
-export_type([opt/0]).
-type state() :: opt() | [{module(), any()}].
-export_type([state/0]).
-callback init(pid(), any(), cow_http:status(),
cow_http:headers(), map()) -> {ok, any()} | disable.
%% @todo Make fin | nofin its own type.
-callback handle(fin | nofin, any(), State)
-> {ok, any(), non_neg_integer(), State}
| {done, non_neg_integer(), State}
when State::any().
-spec init(pid(), any(), cow_http:status(),
cow_http:headers(), State) -> State when State::state().
init(_, _, _, _, []) ->
[];
init(ReplyTo, StreamRef, Status, Headers, [Handler|Tail]) ->
{Mod, Opts} = case Handler of
Tuple = {_, _} -> Tuple;
Atom -> {Atom, #{}}
end,
case Mod:init(ReplyTo, StreamRef, Status, Headers, Opts) of
{ok, State} -> [{Mod, State}|init(ReplyTo, StreamRef, Status, Headers, Tail)];
disable -> init(ReplyTo, StreamRef, Status, Headers, Tail)
end.
-spec handle(fin | nofin, any(), State) -> {ok, non_neg_integer(), State} when State::state().
handle(IsFin, Data, State) ->
handle(IsFin, Data, State, 0, []).
handle(_, _, [], Flow, Acc) ->
{ok, Flow, lists:reverse(Acc)};
handle(IsFin, Data0, [{Mod, State0}|Tail], Flow, Acc) ->
case Mod:handle(IsFin, Data0, State0) of
{ok, Data, Inc, State} ->
handle(IsFin, Data, Tail, Flow + Inc, [{Mod, State}|Acc]);
{done, Inc, State} ->
{ok, Flow + Inc, lists:reverse([{Mod, State}|Acc], Tail)}
end.
-spec check_option(list()) -> ok | error.
check_option([]) ->
error;
check_option(Opt) ->
check_option1(Opt).
check_option1([]) ->
ok;
check_option1([Atom|Tail]) when is_atom(Atom) ->
check_option1(Tail);
check_option1([{Atom, #{}}|Tail]) when is_atom(Atom) ->
check_option1(Tail);
check_option1(_) ->
error.

668
gun/src/gun_cookies.erl Normal file
View File

@ -0,0 +1,668 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_cookies).
-export([add_cookie_header/5]).
-export([domain_match/2]).
-export([gc/1]).
-export([path_match/2]).
-export([query/2]).
-export([session_gc/1]).
-export([set_cookie/5]).
-export([set_cookie_header/7]).
-ifdef(TEST).
-export([wpt_http_state_test_files/1]). %% Also used in rfc6265bis_SUITE.
-endif.
-type store_state() :: any().
-type store() :: {module(), store_state()}.
-export_type([store/0]).
-type cookie() :: #{
name := binary(),
value := binary(),
domain := binary(),
path := binary(),
creation_time := calendar:datetime(),
last_access_time := calendar:datetime(),
expiry_time := calendar:datetime() | infinity,
persistent := boolean(),
host_only := boolean(),
secure_only := boolean(),
http_only := boolean(),
same_site := default | none | strict | lax
}.
-export_type([cookie/0]).
-callback init(any()) -> store().
-callback query(State, uri_string:uri_map())
-> {ok, [gun_cookies:cookie()], State}
when State::store_state().
-callback set_cookie_secure_match(store_state(), #{
name := binary(),
% secure_only := true,
domain := binary(),
path := binary()
}) -> match | nomatch.
-callback set_cookie_get_exact_match(State, #{
name := binary(),
domain := binary(),
host_only := boolean(),
path := binary()
}) -> {ok, cookie(), State} | error
when State::store_state().
-callback store(State, cookie())
-> {ok, State} | {error, any()}
when State::store_state().
-callback gc(State)
-> {ok, State}
when State::store_state().
-callback session_gc(State)
-> {ok, State}
when State::store_state().
-spec add_cookie_header(binary(), iodata(), iodata(), Headers, Store)
-> {Headers, Store} when Headers :: [{binary(), iodata()}], Store :: undefined | store().
add_cookie_header(_, _, _, Headers, Store=undefined) ->
{Headers, Store};
add_cookie_header(Scheme, Authority, PathWithQs, Headers0, Store0) ->
#{
host := Host,
path := Path
} = uri_string:parse([Scheme, <<"://">>, Authority, PathWithQs]),
URIMap = uri_string:normalize(#{
scheme => Scheme,
host => iolist_to_binary(Host),
path => iolist_to_binary(Path)
}, [return_map]),
{ok, Cookies0, Store} = query(Store0, URIMap),
Headers = case Cookies0 of
[] ->
Headers0;
_ ->
Cookies = [{Name, Value} || #{name := Name, value := Value} <- Cookies0],
%% We put cookies at the end of the headers list as it's the least important header.
Headers0 ++ [{<<"cookie">>, cow_cookie:cookie(Cookies)}]
end,
{Headers, Store}.
-spec domain_match(binary(), binary()) -> boolean().
domain_match(String, String) ->
true;
domain_match(String, DomainString) ->
SkipLen = byte_size(String) - byte_size(DomainString) - 1,
case String of
<<_:SkipLen/unit:8, $., DomainString/binary>> ->
case inet:parse_strict_address(binary_to_list(String)) of
{ok, _} ->
false;
{error, einval} ->
true
end;
_ ->
false
end.
-spec gc(Store) -> {ok, Store} when Store::store().
gc({Mod, State0}) ->
{ok, State} = Mod:gc(State0),
{ok, {Mod, State}}.
-spec path_match(binary(), binary()) -> boolean().
path_match(Path, Path) ->
true;
path_match(ReqPath, CookiePath) ->
Len = byte_size(CookiePath),
CookieLast = binary:last(CookiePath),
case ReqPath of
<<CookiePath:Len/binary, _/bits>> when CookieLast =:= $/ ->
true;
<<CookiePath:Len/binary, $/, _/bits>> ->
true;
_ ->
false
end.
-ifdef(TEST).
path_match_test_() ->
Tests = [
{<<"/">>, <<"/">>, true},
{<<"/path/to/resource">>, <<"/path/to/resource">>, true},
{<<"/path/">>, <<"/path/">>, true},
{<<"/path/to/resource">>, <<"/path/">>, true},
{<<"/path/to/resource">>, <<"/path">>, true},
{<<"/path/to/resource">>, <<"/path/to/">>, true},
{<<"/path/to/resource">>, <<"/path/to">>, true},
{<<"/path/to/resource">>, <<"/pa">>, false},
{<<"/path/to/resource">>, <<"/pat">>, false},
{<<"/path/to/resource">>, <<"/path/to/r">>, false},
{<<"/abc">>, <<"/def">>, false}
],
[{iolist_to_binary(io_lib:format("(~p,~p)", [PA, PB])),
fun() -> Res = path_match(PA, PB) end}
|| {PA, PB, Res} <- Tests].
-endif.
%% @todo The given URI must be normalized.
-spec query(Store, uri_string:uri_map())
-> {ok, [cookie()], Store}
when Store::store().
query({Mod, State0}, URI) ->
{ok, Cookies0, State} = Mod:query(State0, URI),
Cookies = lists:sort(fun
(#{path := P, creation_time := CTA}, #{path := P, creation_time := CTB}) ->
CTA =< CTB;
(#{path := PA}, #{path := PB}) ->
PA > PB
end, Cookies0),
{ok, Cookies, {Mod, State}}.
-spec session_gc(Store) -> {ok, Store} when Store::store().
session_gc({Mod, State0}) ->
{ok, State} = Mod:session_gc(State0),
{ok, {Mod, State}}.
%% @todo The given URI must be normalized.
-spec set_cookie(Store, uri_string:uri_map(), binary(), binary(), cow_cookie:cookie_attrs())
-> {ok, Store} | {error, any()} when Store::store().
set_cookie(_, _, Name, Value, _) when byte_size(Name) + byte_size(Value) > 4096 ->
{error, larger_than_4096_bytes};
set_cookie(Store, URI=#{host := Host}, Name, Value, Attrs) ->
%% This is where we would add a feature to block cookies (like a blacklist).
CurrentTime = erlang:universaltime(),
Cookie0 = #{
name => Name,
value => Value,
creation_time => CurrentTime,
last_access_time => CurrentTime
},
Cookie = case Attrs of
#{max_age := ExpiryTime} ->
Cookie0#{
persistent => true,
expiry_time => ExpiryTime
};
#{expires := ExpiryTime} ->
Cookie0#{
persistent => true,
expiry_time => ExpiryTime
};
_ ->
Cookie0#{
persistent => false,
expiry_time => infinity
}
end,
Domain0 = maps:get(domain, Attrs, <<>>),
Domain = case gun_public_suffix:match(Domain0) of
false ->
Domain0;
true when Host =:= Domain0 ->
<<>>;
true ->
{error, domain_is_public_suffix}
end,
case Domain of
<<>> ->
set_cookie(Store, URI, Attrs, Cookie#{
host_only => true,
domain => Host
});
Error = {error, _} ->
Error;
_ ->
case domain_match(Host, Domain) of
true ->
set_cookie(Store, URI, Attrs, Cookie#{
host_only => false,
domain => Domain
});
false ->
{error, domain_match_failed}
end
end.
set_cookie(Store, URI, Attrs, Cookie0) ->
Cookie1 = case Attrs of
#{path := Path} ->
Cookie0#{path => Path};
_ ->
Cookie0#{path => default_path(URI)}
end,
SecureOnly = maps:get(secure, Attrs, false),
case {SecureOnly, maps:get(scheme, URI)} of
{true, <<"http">>} ->
{error, secure_scheme_only};
_ ->
Cookie = Cookie1#{
secure_only => SecureOnly,
http_only => maps:get(http_only, Attrs, false)
},
%% This is where we would drop cookies from non-HTTP APIs.
set_cookie1(Store, URI, Attrs, Cookie)
end.
default_path(#{path := Path}) ->
case string:split(Path, <<"/">>, trailing) of
[_] -> <<"/">>;
[<<>>, _] -> <<"/">>;
[DefaultPath, _] -> DefaultPath
end;
default_path(_) ->
<<"/">>.
set_cookie1(Store, URI=#{scheme := <<"http">>}, Attrs, Cookie=#{secure_only := false}) ->
Match = maps:with([name, domain, path], Cookie),
case set_cookie_secure_match(Store, Match) of
match ->
{error, secure_cookie_matches};
nomatch ->
set_cookie2(Store, URI, Attrs, Cookie)
end;
set_cookie1(Store, URI, Attrs, Cookie) ->
set_cookie2(Store, URI, Attrs, Cookie).
set_cookie_secure_match({Mod, State}, Match) ->
Mod:set_cookie_secure_match(State, Match).
set_cookie2(Store, _URI, Attrs, Cookie0) ->
Cookie = Cookie0#{same_site => maps:get(same_site, Attrs, default)},
%% This is where we would perform the same-site checks.
%%
%% It seems that an option would need to be added to Gun
%% in order to define the "site for cookies" value. It is
%% not the same as the site identified by the URI. Although
%% I do wonder if in the case of server push we may consider
%% the requested URI to be the "site for cookies", at least
%% by default.
%%
%% The URI argument will be used if/when the above gets
%% implemented.
set_cookie3(Store, Attrs, Cookie).
set_cookie3(Store, Attrs, Cookie=#{name := Name,
host_only := HostOnly, secure_only := SecureOnly}) ->
Path = maps:get(path, Attrs, undefined),
case Name of
<<"__Secure-",_/bits>> when not SecureOnly ->
{error, name_prefix_secure_requires_secure_only};
<<"__Host-",_/bits>> when not SecureOnly ->
{error, name_prefix_host_requires_secure_only};
<<"__Host-",_/bits>> when not HostOnly ->
{error, name_prefix_host_requires_host_only};
<<"__Host-",_/bits>> when Path =/= <<"/">> ->
{error, name_prefix_host_requires_top_level_path};
_ ->
set_cookie_store(Store, Cookie)
end.
set_cookie_store(Store0, Cookie) ->
Match = maps:with([name, domain, host_only, path], Cookie),
case set_cookie_get_exact_match(Store0, Match) of
{ok, #{creation_time := CreationTime}, Store} ->
%% This is where we would reject a new non-HTTP cookie
%% if the OldCookie has http_only set to true.
store(Store, Cookie#{creation_time => CreationTime});
error ->
store(Store0, Cookie)
end.
set_cookie_get_exact_match({Mod, State0}, Match) ->
case Mod:set_cookie_get_exact_match(State0, Match) of
{ok, Cookie, State} ->
{ok, Cookie, {Mod, State}};
Error ->
Error
end.
store({Mod, State0}, Cookie) ->
case Mod:store(State0, Cookie) of
{ok, State} ->
{ok, {Mod, State}};
%% @todo Is this return value useful? Can't it just return {ok, State}?
Error ->
Error
end.
-spec set_cookie_header(binary(), iodata(), iodata(), cow_http:status(),
Headers, Store, #{cookie_ignore_informational := boolean()})
-> {Headers, Store} when Headers :: [{binary(), iodata()}], Store :: undefined | store().
%% Don't set cookies when cookie store isn't configured.
set_cookie_header(_, _, _, _, _, Store=undefined, _) ->
Store;
%% Ignore cookies set on informational responses when configured to do so.
%% This includes cookies set to Websocket upgrade responses!
set_cookie_header(_, _, _, Status, _, Store, #{cookie_ignore_informational := true})
when Status >= 100, Status =< 199 ->
Store;
set_cookie_header(Scheme, Authority, PathWithQs, _, Headers, Store0, _) ->
#{host := Host, path := Path} = uri_string:parse([Scheme, <<"://">>, Authority, PathWithQs]),
URIMap = uri_string:normalize(#{
scheme => Scheme,
host => iolist_to_binary(Host),
path => iolist_to_binary(Path)
}, [return_map]),
SetCookies = [SC || {<<"set-cookie">>, SC} <- Headers],
lists:foldl(fun(SC, Store1) ->
case cow_cookie:parse_set_cookie(SC) of
{ok, N, V, A} ->
case set_cookie(Store1, URIMap, N, V, A) of
{ok, Store} -> Store;
{error, _} -> Store1
end;
ignore ->
Store1
end
end, Store0, SetCookies).
-ifdef(TEST).
gc_test() ->
URIMap = #{scheme => <<"http">>, host => <<"example.org">>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a cookie that expires in 2 seconds. GC. Cookie can be retrieved.
{ok, N0, V0, A0} = cow_cookie:parse_set_cookie(<<"a=b; Path=/; Max-Age=2">>),
{ok, Store1} = set_cookie(Store0, URIMap, N0, V0, A0),
{ok, Store2} = gc(Store1),
{ok, [_], _} = query(Store2, URIMap),
%% Wait 3 seconds. GC. Cookie was removed.
timer:sleep(3000),
{ok, Store} = gc(Store2),
{ok, [], _} = query(Store, URIMap),
ok.
gc_expiry_time_infinity_test() ->
URIMap = #{scheme => <<"http">>, host => <<"example.org">>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a session cookie. GC. Cookie can be retrieved.
{ok, N0, V0, A0} = cow_cookie:parse_set_cookie(<<"a=b; Path=/">>),
{ok, Store1} = set_cookie(Store0, URIMap, N0, V0, A0),
{ok, Store} = gc(Store1),
{ok, [_], _} = query(Store, URIMap),
ok.
session_gc_test() ->
URIMap = #{scheme => <<"http">>, host => <<"example.org">>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a persistent and a session cookie. GC session cookies. Only persistent can be retrieved.
{ok, N0, V0, A0} = cow_cookie:parse_set_cookie(<<"s=s; Path=/">>),
{ok, Store1} = set_cookie(Store0, URIMap, N0, V0, A0),
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"p=p; Path=/; Max-Age=999">>),
{ok, Store2} = set_cookie(Store1, URIMap, N1, V1, A1),
{ok, Store} = session_gc(Store2),
{ok, [#{name := <<"p">>}], _} = query(Store, URIMap),
ok.
%% Most of the tests for this module are converted from the
%% Web platform test suite. At the time of writing they could
%% be found at https://github.com/web-platform-tests/wpt/tree/master/cookies
%%
%% Some of the tests use files from wpt directly, namely the
%% http-state ones. They are copied to the test/wpt/cookies directory.
-define(HOST, "web-platform.test").
%% WPT: domain/domain-attribute-host-with-and-without-leading-period
%% WPT: domain/domain-attribute-host-with-leading-period
wpt_domain_with_and_without_leading_period_test() ->
URIMap = #{scheme => <<"http">>, host => <<?HOST>>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a cookie with a leading period in the domain. Cookie can be retrieved.
{ok, N0, V0, A0} = cow_cookie:parse_set_cookie(<<"a=b; Path=/; Domain=." ?HOST>>),
{ok, Store1} = set_cookie(Store0, URIMap, N0, V0, A0),
{ok, [#{value := <<"b">>}], _} = query(Store1, URIMap),
{ok, [#{value := <<"b">>}], _} = query(Store1, URIMap#{host => <<"sub." ?HOST>>}),
%% Add a cookie without a leading period in the domain. Overrides the existing cookie.
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"a=c; Path=/; Domain=" ?HOST>>),
{ok, Store} = set_cookie(Store1, URIMap, N1, V1, A1),
{ok, [#{value := <<"c">>}], _} = query(Store, URIMap),
{ok, [#{value := <<"c">>}], _} = query(Store, URIMap#{host => <<"sub." ?HOST>>}),
ok.
%% WPT: domain/domain-attribute-matches-host
wpt_domain_matches_host_test() ->
URIMap = #{scheme => <<"http">>, host => <<?HOST>>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a cookie without a leading period in the domain. Cookie can be retrieved.
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"a=c; Path=/; Domain=" ?HOST>>),
{ok, Store} = set_cookie(Store0, URIMap, N1, V1, A1),
{ok, [#{value := <<"c">>}], _} = query(Store, URIMap),
{ok, [#{value := <<"c">>}], _} = query(Store, URIMap#{host => <<"sub." ?HOST>>}),
ok.
%% WPT: domain/domain-attribute-missing
wpt_domain_missing_test() ->
URIMap = #{scheme => <<"http">>, host => <<?HOST>>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a cookie without a domain attribute. Cookie is not sent on subdomains.
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"a=c; Path=/">>),
{ok, Store} = set_cookie(Store0, URIMap, N1, V1, A1),
{ok, [#{value := <<"c">>}], _} = query(Store, URIMap),
{ok, [], _} = query(Store, URIMap#{host => <<"sub." ?HOST>>}),
ok.
%% WPT: http-state/*-tests
wpt_http_state_test_files() ->
wpt_http_state_test_files("test/").
wpt_http_state_test_files(TestPath) ->
filelib:wildcard(TestPath ++ "wpt/cookies/*-test") -- [
TestPath ++ "wpt/cookies/attribute0023-test", %% Doesn't match the spec (path override).
TestPath ++ "wpt/cookies/disabled-chromium0020-test", %% Maximum cookie name of 4096 characters.
TestPath ++ "wpt/cookies/disabled-chromium0022-test" %% Nonsense.
].
wpt_http_state_test_() ->
URIMap0 = #{scheme => <<"http">>, host => <<"home.example.org">>, path => <<"/cookie-parser">>},
TestFiles = wpt_http_state_test_files(),
[{F, fun() ->
{ok, Test} = file:read_file(F),
%% We don't want the final empty line.
Lines = lists:reverse(tl(lists:reverse(string:split(Test, <<"\n">>, all)))),
{Store, URIMap2} = lists:foldl(fun
(<<"Set-Cookie: ",SetCookie/bits>>, Acc={Store0, URIMap1}) ->
case cow_cookie:parse_set_cookie(SetCookie) of
{ok, N, V, A} ->
%% We use the URIMap that corresponds to the request.
case set_cookie(Store0, URIMap0, N, V, A) of
{ok, Store1} -> {Store1, URIMap1};
{error, _} -> Acc
end;
ignore ->
Acc
end;
(<<"Set-Cookie:">>, Acc) ->
Acc;
(<<"Location: ",Location/bits>>, {Store0, URIMap1}) ->
{Store0, maps:merge(URIMap1, uri_string:normalize(Location, [return_map]))}
end, {gun_cookies_list:init(), URIMap0}, Lines),
%% We must change the URI if it wasn't already changed by the test.
URIMap = case URIMap2 of
URIMap0 -> maps:merge(URIMap0, uri_string:normalize(<<"/cookie-parser-result">>, [return_map]));
_ -> URIMap2
end,
{ok, Cookies, _} = query(Store, URIMap),
case file:read_file(iolist_to_binary(string:replace(F, <<"-test">>, <<"-expected">>))) of
{ok, ExpectedFile} when ExpectedFile =:= <<>>; ExpectedFile =:= <<"\n">> ->
[] = Cookies,
ok;
{ok, <<"Cookie: ",CookiesBin0/bits>>} ->
%% We only care about the first line.
[CookiesBin, <<>>|_] = string:split(CookiesBin0, <<"\n">>, all),
CookiesBin = iolist_to_binary(cow_cookie:cookie(
[{Name, Value} || #{name := Name, value := Value} <- Cookies])),
ok
end
end} || F <- TestFiles].
%% WPT: path/default
wpt_path_default_test() ->
URIMap = #{scheme => <<"http">>, host => <<?HOST>>, path => <<"/path/to/resource">>},
Store0 = gun_cookies_list:init(),
%% Add a cookie without a path attribute.
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"cookies-path-default=1">>),
{ok, Store1} = set_cookie(Store0, URIMap, N1, V1, A1),
%% Confirm the cookie was stored with the proper default path,
%% and gets sent for the same path, other resources at the same level or child paths.
{ok, [#{path := <<"/path/to">>}], _} = query(Store1, URIMap),
{ok, [#{path := <<"/path/to">>}], _} = query(Store1, URIMap#{path => <<"/path/to/other">>}),
{ok, [#{path := <<"/path/to">>}], _} = query(Store1, URIMap#{path => <<"/path/to/resource/sub">>}),
%% Confirm that the cookie cannot be retrieved for parent or unrelated paths.
{ok, [], _} = query(Store1, URIMap#{path => <<"/path">>}),
{ok, [], _} = query(Store1, URIMap#{path => <<"/path/toon">>}),
{ok, [], _} = query(Store1, URIMap#{path => <<"/">>}),
%% Expire the cookie.
{ok, N2, V2, A2} = cow_cookie:parse_set_cookie(<<"cookies-path-default=1; Max-Age=0">>),
{ok, Store} = set_cookie(Store1, URIMap, N2, V2, A2),
{ok, [], _} = query(Store, URIMap),
{ok, [], _} = query(Store, URIMap#{path => <<"/path/to/other">>}),
{ok, [], _} = query(Store, URIMap#{path => <<"/path/to/resource/sub">>}),
ok.
%% WPT: path/match
wpt_path_match_test_() ->
URIMap = #{
scheme => <<"http">>,
host => <<?HOST>>,
path => <<"/cookies/resources/echo-cookie.html">>
},
MatchTests = [
<<"/">>,
<<"match.html">>,
<<"cookies">>,
<<"/cookies">>,
<<"/cookies/">>,
<<"/cookies/resources/echo-cookie.html">>
],
NegTests = [
<<"/cook">>,
<<"/w/">>
],
[{P, fun() ->
{ok, N1, V1, A1} = cow_cookie:parse_set_cookie(<<"a=b; Path=",P/binary>>),
{ok, Store0} = set_cookie(gun_cookies_list:init(), URIMap, N1, V1, A1),
{ok, [#{name := <<"a">>}], _} = query(Store0, URIMap),
{ok, N2, V2, A2} = cow_cookie:parse_set_cookie(<<"a=b; Max-Age=0; Path=",P/binary>>),
{ok, Store} = set_cookie(Store0, URIMap, N2, V2, A2),
{ok, [], _} = query(Store, URIMap)
end} || P <- MatchTests]
++
[{P, fun() ->
{ok, N, V, A} = cow_cookie:parse_set_cookie(<<"a=b; Path=",P/binary>>),
{ok, Store} = set_cookie(gun_cookies_list:init(), URIMap, N, V, A),
{ok, [], _} = query(Store, URIMap)
end} || P <- NegTests].
%% WPT: prefix/__host.header
wpt_prefix_host_test_() ->
Tests = [
{<<"http">>, <<"__Host-foo=bar; Path=/;">>, false},
{<<"http">>, <<"__Host-foo=bar; Path=/;domain=" ?HOST>>, false},
{<<"http">>, <<"__Host-foo=bar; Path=/;Max-Age=10">>, false},
{<<"http">>, <<"__Host-foo=bar; Path=/;HttpOnly">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/;">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/;domain=" ?HOST>>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/;Max-Age=10">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/;HttpOnly">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; ">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; domain=" ?HOST>>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; Max-Age=10">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; HttpOnly">>, false},
{<<"http">>, <<"__Host-foo=bar; Secure; Path=/cookies/resources/list.py">>, false},
{<<"https">>, <<"__Host-foo=bar; Path=/;">>, false},
{<<"https">>, <<"__Host-foo=bar; Path=/;Max-Age=10">>, false},
{<<"https">>, <<"__Host-foo=bar; Path=/;HttpOnly">>, false},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/;">>, true},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/;Max-Age=10">>, true},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/;HttpOnly">>, true},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; ">>, false},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; Max-Age=10">>, false},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/; Domain=" ?HOST "; HttpOnly">>, false},
{<<"https">>, <<"__Host-foo=bar; Secure; Path=/cookies/resources/list.py">>, false}
],
wpt_prefix_common(Tests, <<"__Host-foo">>).
%% WPT: prefix/__secure.header
wpt_prefix_secure_test_() ->
Tests = [
{<<"http">>, <<"__Secure-foo=bar; Path=/;">>, false},
{<<"http">>, <<"__Secure-foo=bar; Path=/;domain=" ?HOST>>, false},
{<<"http">>, <<"__Secure-foo=bar; Path=/;Max-Age=10">>, false},
{<<"http">>, <<"__Secure-foo=bar; Path=/;HttpOnly">>, false},
{<<"http">>, <<"__Secure-foo=bar; Secure; Path=/;">>, false},
{<<"http">>, <<"__Secure-foo=bar; Secure; Path=/;domain=" ?HOST>>, false},
{<<"http">>, <<"__Secure-foo=bar; Secure; Path=/;Max-Age=10">>, false},
{<<"http">>, <<"__Secure-foo=bar; Secure; Path=/;HttpOnly">>, false},
{<<"https">>, <<"__Secure-foo=bar; Path=/;">>, false},
{<<"https">>, <<"__Secure-foo=bar; Path=/;Max-Age=10">>, false},
{<<"https">>, <<"__Secure-foo=bar; Path=/;HttpOnly">>, false},
{<<"https">>, <<"__Secure-foo=bar; Secure; Path=/;">>, true},
{<<"https">>, <<"__Secure-foo=bar; Secure; Path=/;Max-Age=10">>, true},
{<<"https">>, <<"__Secure-foo=bar; Secure; Path=/;HttpOnly">>, true}
%% Missing two SameSite cases from prefix/__secure.header.https. (Not implemented.)
],
wpt_prefix_common(Tests, <<"__Secure-foo">>).
wpt_prefix_common(Tests, Name) ->
URIMap0 = #{
host => <<?HOST>>,
path => <<"/cookies/resources/set.py">>
},
[{<<S/binary," ",H/binary>>, fun() ->
URIMap1 = URIMap0#{scheme => S},
{ok, N, V, A} = cow_cookie:parse_set_cookie(H),
case set_cookie(gun_cookies_list:init(), URIMap1, N, V, A) of
{ok, Store} when Res =:= true ->
URIMap = URIMap1#{path => <<"/cookies/resources/list.py">>},
{ok, [#{name := Name}], _} = query(Store, URIMap),
ok;
{error, _} ->
ok
end
end} || {S, H, Res} <- Tests].
%% WPT: samesite-none-secure/ (Not implemented.)
%% WPT: samesite/ (Not implemented.)
wpt_secure_https_test() ->
URIMap = #{
scheme => <<"https">>,
host => <<?HOST>>,
path => <<"/cookies/secure/any.html">>
},
{ok, N, V, A} = cow_cookie:parse_set_cookie(<<"secure_from_secure_http=1; Secure; Path=/">>),
{ok, Store} = set_cookie(gun_cookies_list:init(), URIMap, N, V, A),
{ok, [#{name := <<"secure_from_secure_http">>}], _} = query(Store, URIMap),
ok.
wpt_secure_http_test() ->
URIMap = #{
scheme => <<"http">>,
host => <<?HOST>>,
path => <<"/cookies/secure/any.html">>
},
{ok, N, V, A} = cow_cookie:parse_set_cookie(<<"secure_from_nonsecure_http=1; Secure; Path=/">>),
{error, secure_scheme_only} = set_cookie(gun_cookies_list:init(), URIMap, N, V, A),
ok.
%% WPT: secure/set-from-ws* (Anything special required?)
-endif.

144
gun/src/gun_cookies_list.erl Normal file
View File

@ -0,0 +1,144 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
%% A reference cookie store implemented as a list of cookies.
%% This cookie store cannot be shared between connections.
-module(gun_cookies_list).
-behavior(gun_cookies).
-export([init/0]).
-export([init/1]).
-export([query/2]).
-export([set_cookie_secure_match/2]).
-export([set_cookie_get_exact_match/2]).
-export([store/2]).
-export([gc/1]).
-export([session_gc/1]).
-type state() :: #{
cookies := [gun_cookies:cookie()]
%% @todo max_cookies_per_domain => non_neg_integer() | infinity,
%% @todo max_cookies => non_neg_integer() | infinity
}.
-type opts() :: #{
}.
-export_type([opts/0]).
-spec init() -> {?MODULE, state()}.
init() ->
init(#{}).
-spec init(opts()) -> {?MODULE, state()}.
init(_Opts) ->
{?MODULE, #{cookies => []}}.
-spec query(State, uri_string:uri_map())
-> {ok, [gun_cookies:cookie()], State}
when State::state().
query(State=#{cookies := Cookies}, URI) ->
CurrentTime = erlang:universaltime(),
query(State, URI, Cookies, CurrentTime, [], []).
query(State, _, [], _, CookieList, Cookies) ->
{ok, CookieList, State#{cookies => Cookies}};
query(State, URI=#{scheme := Scheme, host := Host, path := Path},
[Cookie|Tail], CurrentTime, CookieList, Acc) ->
Match0 = case Cookie of
#{host_only := true, domain := Host} ->
true;
#{host_only := false, domain := Domain} ->
gun_cookies:domain_match(Host, Domain);
_ ->
false
end,
Match1 = Match0 andalso
gun_cookies:path_match(Path, maps:get(path, Cookie)),
Match = Match1 andalso
case {Cookie, Scheme} of
{#{secure_only := true}, <<"https">>} -> true;
{#{secure_only := false}, _} -> true;
_ -> false
end,
%% This is where we would check the http_only flag should
%% we want to implement a non-HTTP interface.
%% This is where we would check for same-site/cross-site.
case Match of
true ->
UpdatedCookie = Cookie#{last_access_time => CurrentTime},
query(State, URI, Tail, CurrentTime,
[UpdatedCookie|CookieList],
[UpdatedCookie|Acc]);
false ->
query(State, URI, Tail, CurrentTime, CookieList, [Cookie|Acc])
end.
-spec set_cookie_secure_match(state(), #{
name := binary(),
% secure_only := true,
domain := binary(),
path := binary()
}) -> match | nomatch.
set_cookie_secure_match(#{cookies := Cookies},
#{name := Name, domain := Domain, path := Path}) ->
Result = [Cookie || Cookie=#{name := CookieName, secure_only := true} <- Cookies,
CookieName =:= Name,
gun_cookies:domain_match(Domain, maps:get(domain, Cookie))
orelse gun_cookies:domain_match(maps:get(domain, Cookie), Domain),
gun_cookies:path_match(Path, maps:get(path, Cookie))],
case Result of
[] -> nomatch;
_ -> match
end.
-spec set_cookie_get_exact_match(State, #{
name := binary(),
domain := binary(),
host_only := boolean(),
path := binary()
}) -> {ok, gun_cookies:cookie(), State} | error when State::state().
set_cookie_get_exact_match(State=#{cookies := Cookies0}, Match) ->
Result = [Cookie || Cookie <- Cookies0,
Match =:= maps:with([name, domain, host_only, path], Cookie)],
Cookies = [Cookie || Cookie <- Cookies0,
Match =/= maps:with([name, domain, host_only, path], Cookie)],
case Result of
[] -> error;
[Cookie] -> {ok, Cookie, State#{cookies => Cookies}}
end.
-spec store(State, gun_cookies:cookie())
-> {ok, State} | {error, any()}
when State::state().
store(State=#{cookies := Cookies}, NewCookie=#{expiry_time := ExpiryTime}) ->
CurrentTime = erlang:universaltime(),
if
%% Do not store cookies with an expiry time in the past.
ExpiryTime =/= infinity, CurrentTime >= ExpiryTime ->
{ok, State};
true ->
{ok, State#{cookies => [NewCookie|Cookies]}}
end.
-spec gc(State) -> {ok, State} when State::state().
gc(State=#{cookies := Cookies0}) ->
CurrentTime = erlang:universaltime(),
Cookies = [C || C=#{expiry_time := ExpiryTime} <- Cookies0,
(ExpiryTime =:= infinity) orelse (ExpiryTime >= CurrentTime)],
{ok, State#{cookies => Cookies}}.
-spec session_gc(State) -> {ok, State} when State::state().
session_gc(State=#{cookies := Cookies0}) ->
Cookies = [C || C=#{persistent := true} <- Cookies0],
{ok, State#{cookies => Cookies}}.

33
gun/src/gun_data_h.erl Normal file
View File

@ -0,0 +1,33 @@
%% Copyright (c) 2017-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_data_h).
-behavior(gun_content_handler).
-export([init/5]).
-export([handle/3]).
-record(state, {
reply_to :: pid(),
stream_ref :: gun:stream_ref()
}).
-spec init(pid(), gun:stream_ref(), _, _, _) -> {ok, #state{}}.
init(ReplyTo, StreamRef, _, _, _) ->
{ok, #state{reply_to=ReplyTo, stream_ref=StreamRef}}.
-spec handle(fin | nofin, binary(), State) -> {done, 1, State} when State::#state{}.
handle(IsFin, Data, State=#state{reply_to=ReplyTo, stream_ref=StreamRef}) ->
ReplyTo ! {gun_data, self(), StreamRef, IsFin, Data},
{done, 1, State}.

129
gun/src/gun_default_event_h.erl Normal file
View File

@ -0,0 +1,129 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_default_event_h).
-behavior(gun_event).
-export([init/2]).
-export([domain_lookup_start/2]).
-export([domain_lookup_end/2]).
-export([connect_start/2]).
-export([connect_end/2]).
-export([tls_handshake_start/2]).
-export([tls_handshake_end/2]).
-export([request_start/2]).
-export([request_headers/2]).
-export([request_end/2]).
-export([push_promise_start/2]).
-export([push_promise_end/2]).
-export([response_start/2]).
-export([response_inform/2]).
-export([response_headers/2]).
-export([response_trailers/2]).
-export([response_end/2]).
-export([ws_upgrade/2]).
-export([ws_recv_frame_start/2]).
-export([ws_recv_frame_header/2]).
-export([ws_recv_frame_end/2]).
-export([ws_send_frame_start/2]).
-export([ws_send_frame_end/2]).
-export([protocol_changed/2]).
-export([origin_changed/2]).
-export([cancel/2]).
-export([disconnect/2]).
-export([terminate/2]).
init(_EventData, State) ->
State.
domain_lookup_start(_EventData, State) ->
State.
domain_lookup_end(_EventData, State) ->
State.
connect_start(_EventData, State) ->
State.
connect_end(_EventData, State) ->
State.
tls_handshake_start(_EventData, State) ->
State.
tls_handshake_end(_EventData, State) ->
State.
request_start(_EventData, State) ->
State.
request_headers(_EventData, State) ->
State.
request_end(_EventData, State) ->
State.
push_promise_start(_EventData, State) ->
State.
push_promise_end(_EventData, State) ->
State.
response_start(_EventData, State) ->
State.
response_inform(_EventData, State) ->
State.
response_headers(_EventData, State) ->
State.
response_trailers(_EventData, State) ->
State.
response_end(_EventData, State) ->
State.
ws_upgrade(_EventData, State) ->
State.
ws_recv_frame_start(_EventData, State) ->
State.
ws_recv_frame_header(_EventData, State) ->
State.
ws_recv_frame_end(_EventData, State) ->
State.
ws_send_frame_start(_EventData, State) ->
State.
ws_send_frame_end(_EventData, State) ->
State.
protocol_changed(_EventData, State) ->
State.
origin_changed(_EventData, State) ->
State.
cancel(_EventData, State) ->
State.
disconnect(_EventData, State) ->
State.
terminate(_EventData, State) ->
State.

316
gun/src/gun_event.erl Normal file
View File

@ -0,0 +1,316 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_event).
%% init.
-type init_event() :: #{
owner := pid(),
transport := tcp | tls,
origin_scheme := binary(),
origin_host := inet:hostname() | inet:ip_address(),
origin_port := inet:port_number(),
opts := gun:opts()
}.
-export_type([init_event/0]).
-callback init(init_event(), State) -> State.
%% domain_lookup_start/domain_lookup_end.
-type domain_lookup_event() :: #{
host := inet:hostname() | inet:ip_address(),
port := inet:port_number(),
tcp_opts := [gen_tcp:connect_option()],
timeout := timeout(),
lookup_info => gun_tcp:lookup_info(),
error => any()
}.
-export_type([domain_lookup_event/0]).
-callback domain_lookup_start(domain_lookup_event(), State) -> State.
-callback domain_lookup_end(domain_lookup_event(), State) -> State.
%% connect_start/connect_end.
-type connect_event() :: #{
lookup_info := gun_tcp:lookup_info(),
timeout := timeout(),
socket => inet:socket(),
protocol => http | http2 | socks | raw, %% Only when transport is tcp.
error => any()
}.
-export_type([connect_event/0]).
-callback connect_start(connect_event(), State) -> State.
-callback connect_end(connect_event(), State) -> State.
%% tls_handshake_start/tls_handshake_end.
%%
%% These events occur when connecting to a TLS server or when
%% upgrading the connection or stream to use TLS, for example
%% using CONNECT. The stream_ref/reply_to values are only
%% present when the TLS handshake occurs in the scope of a request.
-type tls_handshake_event() :: #{
stream_ref => gun:stream_ref(),
reply_to => pid(),
socket := inet:socket() | ssl:sslsocket() | pid(), %% The socket before/after will be different.
tls_opts := [ssl:tls_client_option()],
timeout := timeout(),
protocol => http | http2 | socks | raw,
error => any()
}.
-export_type([tls_handshake_event/0]).
-callback tls_handshake_start(tls_handshake_event(), State) -> State.
-callback tls_handshake_end(tls_handshake_event(), State) -> State.
%% request_start/request_headers.
-type request_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
function := headers | request | ws_upgrade, %% @todo connect?
method := iodata(),
scheme => binary(),
authority := iodata(),
path => iodata(),
headers := [{binary(), iodata()}]
}.
-export_type([request_start_event/0]).
-callback request_start(request_start_event(), State) -> State.
-callback request_headers(request_start_event(), State) -> State.
%% request_end.
-type request_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}.
-export_type([request_end_event/0]).
-callback request_end(request_end_event(), State) -> State.
%% push_promise_start.
-type push_promise_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}.
-export_type([push_promise_start_event/0]).
-callback push_promise_start(push_promise_start_event(), State) -> State.
%% push_promise_end.
-type push_promise_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
%% No stream is created if we receive the push_promise while
%% in the process of gracefully shutting down the connection.
%% The promised stream is canceled immediately.
promised_stream_ref => gun:stream_ref(),
method := binary(),
uri := binary(),
headers := [{binary(), iodata()}]
}.
-export_type([push_promise_end_event/0]).
-callback push_promise_end(push_promise_end_event(), State) -> State.
%% response_start.
-type response_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}.
-export_type([response_start_event/0]).
-callback response_start(response_start_event(), State) -> State.
%% response_inform/response_headers.
-type response_headers_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
status := non_neg_integer(),
headers := [{binary(), binary()}]
}.
-export_type([response_headers_event/0]).
-callback response_inform(response_headers_event(), State) -> State.
-callback response_headers(response_headers_event(), State) -> State.
%% response_trailers.
-type response_trailers_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
headers := [{binary(), binary()}]
}.
-export_type([response_trailers_event/0]).
-callback response_trailers(response_trailers_event(), State) -> State.
%% response_end.
-type response_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid()
}.
-export_type([response_end_event/0]).
-callback response_end(response_end_event(), State) -> State.
%% ws_upgrade.
%%
%% This event is a signal that the following request and response
%% result from a gun:ws_upgrade/2,3,4 call.
%%
%% There is no corresponding "end" event. Instead, the success is
%% indicated by a protocol_changed event following the informational
%% response.
-type ws_upgrade_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
opts := gun:ws_opts()
}.
-export_type([ws_upgrade_event/0]).
-callback ws_upgrade(ws_upgrade_event(), State) -> State.
%% ws_recv_frame_start.
-type ws_recv_frame_start_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
frag_state := cow_ws:frag_state(),
extensions := cow_ws:extensions()
}.
-export_type([ws_recv_frame_start_event/0]).
-callback ws_recv_frame_start(ws_recv_frame_start_event(), State) -> State.
%% ws_recv_frame_header.
-type ws_recv_frame_header_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
frag_state := cow_ws:frag_state(),
extensions := cow_ws:extensions(),
type := cow_ws:frame_type(),
rsv := cow_ws:rsv(),
len := non_neg_integer(),
mask_key := cow_ws:mask_key()
}.
-export_type([ws_recv_frame_header_event/0]).
-callback ws_recv_frame_header(ws_recv_frame_header_event(), State) -> State.
%% ws_recv_frame_end.
-type ws_recv_frame_end_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
extensions := cow_ws:extensions(),
close_code := undefined | cow_ws:close_code(),
payload := binary()
}.
-export_type([ws_recv_frame_end_event/0]).
-callback ws_recv_frame_end(ws_recv_frame_end_event(), State) -> State.
%% ws_send_frame_start/ws_send_frame_end.
-type ws_send_frame_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
extensions := cow_ws:extensions(),
frame := gun:ws_frame()
}.
-export_type([ws_send_frame_event/0]).
-callback ws_send_frame_start(ws_send_frame_event(), State) -> State.
-callback ws_send_frame_end(ws_send_frame_event(), State) -> State.
%% protocol_changed.
%%
%% This event can occur either following a successful ws_upgrade
%% event, following a successful CONNECT request or a SOCKS tunnel.
-type protocol_changed_event() :: #{
stream_ref => gun:stream_ref(),
protocol := http | http2 | socks | raw | ws
}.
-export_type([protocol_changed_event/0]).
-callback protocol_changed(protocol_changed_event(), State) -> State.
%% origin_changed.
-type origin_changed_event() :: #{
stream_ref => gun:stream_ref(),
type := connect | socks5,
origin_scheme := binary(),
origin_host := inet:hostname() | inet:ip_address(),
origin_port := inet:port_number()
}.
-export_type([origin_changed_event/0]).
-callback origin_changed(origin_changed_event(), State) -> State.
%% cancel.
%%
%% In the case of HTTP/1.1 we cannot actually cancel the stream,
%% we only silence the stream to the user. Further response events
%% may therefore be received and they provide a useful metric as
%% these canceled requests monopolize the connection.
%%
%% For HTTP/2 both the client and the server may cancel streams.
%% Events may still occur for a short time after the cancel.
-type cancel_event() :: #{
stream_ref := gun:stream_ref(),
reply_to := pid(),
endpoint := local | remote,
reason := atom()
}.
-export_type([cancel_event/0]).
-callback cancel(cancel_event(), State) -> State.
%% disconnect.
-type disconnect_event() :: #{
reason := normal | closed | {error, any()}
}.
-export_type([disconnect_event/0]).
-callback disconnect(disconnect_event(), State) -> State.
%% terminate.
-type terminate_event() :: #{
state := not_connected
| domain_lookup | connecting | initial_tls_handshake | tls_handshake
| connected | connected_data_only | connected_ws_only,
reason := normal | shutdown | {shutdown, any()} | any()
}.
-export_type([terminate_event/0]).
-callback terminate(terminate_event(), State) -> State.

1058
gun/src/gun_http.erl Normal file
View File

File diff suppressed because it is too large Load Diff

1601
gun/src/gun_http2.erl Normal file
View File

File diff suppressed because it is too large Load Diff

719
gun/src/gun_pool.erl Normal file
View File

@ -0,0 +1,719 @@
%% Copyright (c) 2021-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_pool).
-behaviour(gen_statem).
%% Pools.
-export([start_pool/3]).
-export([stop_pool/2]).
-export([stop_pool/3]).
% @todo shutdown pool?
-export([info/0]).
-export([info/1]).
-export([info/2]).
-export([await_up/1]).
-export([await_up/2]).
-export([checkout/2]). %% Use responsibly!
%% Requests.
-export([delete/2]).
-export([delete/3]).
-export([get/2]).
-export([get/3]).
-export([head/2]).
-export([head/3]).
-export([options/2]).
-export([options/3]).
-export([patch/2]).
-export([patch/3]).
-export([patch/4]).
-export([post/2]).
-export([post/3]).
-export([post/4]).
-export([put/2]).
-export([put/3]).
-export([put/4]).
%% Generic requests interface.
-export([headers/3]).
-export([headers/4]).
-export([request/4]).
-export([request/5]).
%% Streaming data.
-export([data/3]).
%% Tunneling. (HTTP/2+ only.)
%% @todo -export([connect/2]).
%% @todo -export([connect/3]).
%% @todo -export([connect/4]).
%% Cookies.
%% @todo -export([gc_cookies/1]).
%% @todo -export([session_gc_cookies/1]).
%% Awaiting gun messages.
-export([await/1]).
-export([await/2]).
-export([await/3]).
-export([await_body/1]).
-export([await_body/2]).
-export([await_body/3]).
%% Flushing gun messages.
-export([flush/1]).
%% Streams.
-export([update_flow/2]).
-export([cancel/1]).
-export([stream_info/1]).
%% Websocket. (HTTP/2+ only for upgrade.)
%% -export([ws_upgrade/1]).
%% -export([ws_upgrade/2]).
%% -export([ws_upgrade/3]).
-export([ws_send/2]).
%% -export([ws_send/3]). (HTTP/2+ only.)
%% Internals.
-export([callback_mode/0]).
-export([start_link/3]).
-export([init/1]).
-export([degraded/3]).
-export([operational/3]).
-export([terminate/3]).
-type setup_msg() :: {gun_up, pid(), http | http2 | raw | socks}
| {gun_upgrade, pid(), gun:stream_ref(), [binary()], [{binary(), binary()}]}.
-type opts() :: #{
conn_opts => gun:opts(),
scope => any(),
setup_fun => {fun((pid(), setup_msg(), any()) -> any()), any()},
size => non_neg_integer()
}.
-export_type([opts/0]).
-type pool_stream_ref() :: {pid(), gun:stream_ref()}.
-export_type([pool_stream_ref/0]).
-type error_result() :: {error, pool_not_found | no_connection_available, atom()}. %% @todo {pool_start_error, SupError}
-type request_result() :: {async, pool_stream_ref()}
%% @todo {sync, ...} perhaps with Status, Headers, Body, and Extra info such as intermediate responses.
| error_result().
-type req_opts() :: #{
%% Options common with normal Gun.
flow => pos_integer(),
reply_to => pid(),
% @todo tunnel => stream_ref(),
%% Options specific to pools.
checkout_call_timeout => timeout(),
checkout_retry => [pos_integer()],
scope => any(),
start_pool_if_missing => boolean()
}.
-export_type([req_opts/0]).
-type ws_send_opts() :: #{
authority => iodata(),
%% Options specific to pools.
checkout_call_timeout => timeout(),
checkout_retry => [pos_integer()],
scope => any(),
start_pool_if_missing => boolean()
}.
-export_type([ws_send_opts/0]).
%% @todo tunnel
-type meta() :: #{pid() => #{ws => gun:stream_ref()}}.
-record(state, {
host :: inet:hostname() | inet:ip_address(),
port :: inet:port_number(),
opts :: opts(),
table :: ets:tid(),
conns :: #{pid() => down | {setup, any()} | {up, http | http2 | ws | raw, map()}},
conns_meta = #{} :: meta(),
await_up = [] :: [{pid(), any()}]
}).
%% Pool management.
-spec start_pool(inet:hostname() | inet:ip_address(), inet:port_number(), opts())
-> {ok, pid()} | {error, any()}.
start_pool(Host, Port, Opts) ->
supervisor:start_child(gun_pools_sup, [Host, Port, Opts]).
-spec stop_pool(inet:hostname() | inet:ip_address(), inet:port_number())
-> ok.
stop_pool(Host, Port) ->
stop_pool(Host, Port, #{}).
-type stop_opts() :: #{
scope => any(),
transport => tcp | tls
}.
-spec stop_pool(inet:hostname() | inet:ip_address(), inet:port_number(), stop_opts())
-> ok | {error, pool_not_found, atom()}.
stop_pool(Host, Port, StopOpts) ->
Transport = maps:get(transport, StopOpts, gun:default_transport(Port)),
Authority = gun_http:host_header(Transport, Host, Port),
case get_pool(Authority, StopOpts) of
undefined ->
{error, pool_not_found,
'No pool was found for the given scope and authority.'};
ManagerPid ->
supervisor:terminate_child(gun_pools_sup, ManagerPid)
end.
-spec info() -> [map()].
info() ->
ets:foldl(fun({{Scope, _}, ManagerPid}, Acc) ->
{StateName, Info} = info(ManagerPid),
[Info#{scope => Scope, state => StateName}|Acc]
end, [], gun_pools).
-spec info(pid() | binary()) -> undefined | {degraded | operational, map()}.
info(ManagerPid) when is_pid(ManagerPid) ->
gen_statem:call(ManagerPid, info);
info(Authority) ->
info(Authority, default).
-spec info(binary(), any()) -> undefined | {degraded | operational, map()}.
info(Authority, Scope) ->
case ets:lookup(gun_pools, {Scope, Authority}) of
[] ->
undefined;
[{_, ManagerPid}] ->
gen_statem:call(ManagerPid, info)
end.
-spec await_up(pid() | binary()) -> ok | {error, pool_not_found, atom()}.
await_up(ManagerPid) when is_pid(ManagerPid) ->
gen_statem:call(ManagerPid, await_up, 5000);
await_up(Authority) ->
await_up(Authority, default).
-spec await_up(binary(), any()) -> ok | {error, pool_not_found, atom()}.
await_up(Authority, Scope) ->
case ets:lookup(gun_pools, {Scope, Authority}) of
[] ->
{error, pool_not_found,
'No pool was found for the given scope and authority.'};
[{_, ManagerPid}] ->
gen_statem:call(ManagerPid, await_up, 5000)
end.
-spec checkout(pid(), req_opts() | ws_send_opts()) -> undefined | {pid(), map()}.
checkout(ManagerPid, ReqOpts=#{checkout_retry := Retry}) when is_list(Retry) ->
CallTimeout = maps:get(checkout_call_timeout, ReqOpts, 5000),
case gen_server:call(ManagerPid, {checkout, ReqOpts}, CallTimeout) of
undefined ->
checkout_retry(ManagerPid, ReqOpts, CallTimeout, Retry);
Result ->
Result
end;
checkout(ManagerPid, ReqOpts) ->
CallTimeout = maps:get(checkout_call_timeout, ReqOpts, 5000),
gen_server:call(ManagerPid, {checkout, ReqOpts}, CallTimeout).
%% When the checkout_retry option is used, and the first call resulted
%% in no connection being given out, we wait for the configured amount
%% of time then try again. We loop over the wait times until there is
%% none.
checkout_retry(_, _, _, []) ->
undefined;
checkout_retry(ManagerPid, ReqOpts, CallTimeout, [Wait|Retry]) ->
timer:sleep(Wait),
case gen_server:call(ManagerPid, {checkout, ReqOpts}, CallTimeout) of
undefined ->
checkout_retry(ManagerPid, ReqOpts, CallTimeout, Retry);
Result ->
Result
end.
%% Requests.
-spec delete(iodata(), gun:req_headers()) -> request_result().
delete(Path, Headers) ->
request(<<"DELETE">>, Path, Headers, <<>>).
-spec delete(iodata(), gun:req_headers(), req_opts()) -> request_result().
delete(Path, Headers, ReqOpts) ->
request(<<"DELETE">>, Path, Headers, <<>>, ReqOpts).
-spec get(iodata(), gun:req_headers()) -> request_result().
get(Path, Headers) ->
request(<<"GET">>, Path, Headers, <<>>).
-spec get(iodata(), gun:req_headers(), req_opts()) -> request_result().
get(Path, Headers, ReqOpts) ->
request(<<"GET">>, Path, Headers, <<>>, ReqOpts).
-spec head(iodata(), gun:req_headers()) -> request_result().
head(Path, Headers) ->
request(<<"HEAD">>, Path, Headers, <<>>).
-spec head(iodata(), gun:req_headers(), req_opts()) -> request_result().
head(Path, Headers, ReqOpts) ->
request(<<"HEAD">>, Path, Headers, <<>>, ReqOpts).
-spec options(iodata(), gun:req_headers()) -> request_result().
options(Path, Headers) ->
request(<<"OPTIONS">>, Path, Headers, <<>>).
-spec options(iodata(), gun:req_headers(), req_opts()) -> request_result().
options(Path, Headers, ReqOpts) ->
request(<<"OPTIONS">>, Path, Headers, <<>>, ReqOpts).
-spec patch(iodata(), gun:req_headers()) -> request_result().
patch(Path, Headers) ->
headers(<<"PATCH">>, Path, Headers).
-spec patch(iodata(), gun:req_headers(), iodata() | req_opts()) -> request_result().
patch(Path, Headers, ReqOpts) when is_map(ReqOpts) ->
headers(<<"PATCH">>, Path, Headers, ReqOpts);
patch(Path, Headers, Body) ->
request(<<"PATCH">>, Path, Headers, Body).
-spec patch(iodata(), gun:req_headers(), iodata(), req_opts()) -> request_result().
patch(Path, Headers, Body, ReqOpts) ->
request(<<"PATCH">>, Path, Headers, Body, ReqOpts).
-spec post(iodata(), gun:req_headers()) -> request_result().
post(Path, Headers) ->
headers(<<"POST">>, Path, Headers).
-spec post(iodata(), gun:req_headers(), iodata() | req_opts()) -> request_result().
post(Path, Headers, ReqOpts) when is_map(ReqOpts) ->
headers(<<"POST">>, Path, Headers, ReqOpts);
post(Path, Headers, Body) ->
request(<<"POST">>, Path, Headers, Body).
-spec post(iodata(), gun:req_headers(), iodata(), req_opts()) -> request_result().
post(Path, Headers, Body, ReqOpts) ->
request(<<"POST">>, Path, Headers, Body, ReqOpts).
-spec put(iodata(), gun:req_headers()) -> request_result().
put(Path, Headers) ->
headers(<<"PUT">>, Path, Headers).
-spec put(iodata(), gun:req_headers(), iodata() | req_opts()) -> request_result().
put(Path, Headers, ReqOpts) when is_map(ReqOpts) ->
headers(<<"PUT">>, Path, Headers, ReqOpts);
put(Path, Headers, Body) ->
request(<<"PUT">>, Path, Headers, Body).
-spec put(iodata(), gun:req_headers(), iodata(), req_opts()) -> request_result().
put(Path, Headers, Body, ReqOpts) ->
request(<<"PUT">>, Path, Headers, Body, ReqOpts).
%% Generic requests interface.
%%
%% @todo Accept a TargetURI map as well as a normal Path.
-spec headers(iodata(), iodata(), gun:req_headers()) -> request_result().
headers(Method, Path, Headers) ->
headers(Method, Path, Headers, #{}).
-spec headers(iodata(), iodata(), gun:req_headers(), req_opts()) -> request_result().
headers(Method, Path, Headers, ReqOpts) ->
case get_pool(authority(Headers), ReqOpts) of
undefined ->
{error, pool_not_found,
'No pool was found for the given scope and authority.'};
ManagerPid ->
case checkout(ManagerPid, ReqOpts) of
undefined ->
{error, no_connection_available,
'No connection in the pool with enough capacity available to open a new stream.'};
{ConnPid, _Meta} ->
StreamRef = gun:headers(ConnPid, Method, Path, Headers, ReqOpts),
%% @todo Synchronous mode.
{async, {ConnPid, StreamRef}}
end
end.
-spec request(iodata(), iodata(), gun:req_headers(), iodata()) -> request_result().
request(Method, Path, Headers, Body) ->
request(Method, Path, Headers, Body, #{}).
-spec request(iodata(), iodata(), gun:req_headers(), iodata(), req_opts()) -> request_result().
request(Method, Path, Headers, Body, ReqOpts) ->
case get_pool(authority(Headers), ReqOpts) of
undefined ->
{error, pool_not_found,
'No pool was found for the given scope and authority.'};
ManagerPid ->
case checkout(ManagerPid, ReqOpts) of
undefined ->
{error, no_connection_available,
'No connection in the pool with enough capacity available to open a new stream.'};
{ConnPid, _Meta} ->
StreamRef = gun:request(ConnPid, Method, Path, Headers, Body, ReqOpts),
%% @todo Synchronous mode.
{async, {ConnPid, StreamRef}}
end
end.
%% We require the host to be given in the headers for the time being.
%% @todo Allow passing it in options. Websocket send already does that.
authority(#{<<"host">> := Authority}) ->
Authority;
authority(Headers) ->
{_, Authority} = lists:keyfind(<<"host">>, 1, Headers),
Authority.
-spec get_pool(iolist(), req_opts() | ws_send_opts() | stop_opts()) -> pid() | undefined.
get_pool(Authority0, ReqOpts) ->
Authority = iolist_to_binary(Authority0),
%% @todo Perhaps rename this to temporary.
%% There's two concepts: temporary pool is started and stops
%% when there's no traffic. Dynamic pool simply reduces its number
%% of connections when there's no traffic.
StartPoolIfMissing = maps:get(start_pool_if_missing, ReqOpts, false),
Scope = maps:get(scope, ReqOpts, default),
case ets:lookup(gun_pools, {Scope, Authority}) of
[] when StartPoolIfMissing ->
start_missing_pool(Authority, ReqOpts);
[] ->
undefined;
[{_, ManagerPid}] ->
%% @todo With temporary pool, getting a pid here doesn't mean the pool can be used.
%% Indeed the manager could be in process of stopping. I suppose we must
%% do a check but perhaps it's best to leave that detail to the user
%% (they can easily retry and recreate the pool if necessary).
ManagerPid
end.
start_missing_pool(_Authority, _ReqOpts) ->
undefined.
%% Streaming data.
-spec data(pool_stream_ref(), fin | nofin, iodata()) -> ok.
data({ConnPid, StreamRef}, IsFin, Data) ->
gun:data(ConnPid, StreamRef, IsFin, Data).
%% Awaiting gun messages.
-spec await(pool_stream_ref()) -> gun:await_result().
await({ConnPid, StreamRef}) ->
gun:await(ConnPid, StreamRef).
-spec await(pool_stream_ref(), timeout() | reference()) -> gun:await_result().
await({ConnPid, StreamRef}, MRefOrTimeout) ->
gun:await(ConnPid, StreamRef, MRefOrTimeout).
-spec await(pool_stream_ref(), timeout(), reference()) -> gun:await_result().
await({ConnPid, StreamRef}, Timeout, MRef) ->
gun:await(ConnPid, StreamRef, Timeout, MRef).
-spec await_body(pool_stream_ref()) -> gun:await_body_result().
await_body({ConnPid, StreamRef}) ->
gun:await_body(ConnPid, StreamRef).
-spec await_body(pool_stream_ref(), timeout() | reference()) -> gun:await_body_result().
await_body({ConnPid, StreamRef}, MRefOrTimeout) ->
gun:await_body(ConnPid, StreamRef, MRefOrTimeout).
-spec await_body(pool_stream_ref(), timeout(), reference()) -> gun:await_body_result().
await_body({ConnPid, StreamRef}, Timeout, MRef) ->
gun:await_body(ConnPid, StreamRef, Timeout, MRef).
%% Flushing gun messages.
-spec flush(pool_stream_ref()) -> ok.
flush({ConnPid, _}) ->
gun:flush(ConnPid).
%% Flow control.
-spec update_flow(pool_stream_ref(), pos_integer()) -> ok.
update_flow({ConnPid, StreamRef}, Flow) ->
gun:update_flow(ConnPid, StreamRef, Flow).
%% Cancelling a stream.
-spec cancel(pool_stream_ref()) -> ok.
cancel({ConnPid, StreamRef}) ->
gun:cancel(ConnPid, StreamRef).
%% Information about a stream.
-spec stream_info(pool_stream_ref()) -> {ok, map() | undefined} | {error, not_connected}.
stream_info({ConnPid, StreamRef}) ->
gun:stream_info(ConnPid, StreamRef).
%% Websocket.
-spec ws_send(gun:ws_frame() | [gun:ws_frame()], ws_send_opts()) -> ok | error_result().
ws_send(Frames, WsSendOpts=#{authority := Authority}) ->
case get_pool(Authority, WsSendOpts) of
undefined ->
{error, pool_not_found,
'No pool was found for the given scope and authority.'};
ManagerPid ->
case checkout(ManagerPid, WsSendOpts) of
undefined ->
{error, no_connection_available,
'No connection in the pool with enough capacity available to send Websocket frames.'};
{ConnPid, #{ws := StreamRef}} ->
gun:ws_send(ConnPid, StreamRef, Frames)
end
end.
%% Pool manager internals.
%%
%% The pool manager is responsible for starting connection processes
%% and restarting them as necessary. It also provides a suitable
%% connection process to any caller that needs it.
%%
%% The pool manager installs an event handler into each connection.
%% The event handler is responsible for counting the number of
%% active streams. It updates the gun_pooled_conns ets table
%% whenever a stream begins or ends.
%%
%% A connection is deemed suitable if it is possible to open new
%% streams. How many streams can be open at any one time depends
%% on the protocol. For HTTP/2 the manager process keeps track of
%% the connection's settings to know the maximum. For non-stream
%% based protocols, there is no limit.
%%
%% The connection to be used is otherwise chosen randomly. The
%% first connection that is suitable is returned. There is no
%% need to "give back" the connection to the manager.
%% @todo
%% What should happen if we always fail to reconnect? I suspect we keep the manager
%% around and propagate errors, the same as if there's no more capacity? Perhaps have alarms?
callback_mode() -> state_functions.
start_link(Host, Port, Opts) ->
gen_statem:start_link(?MODULE, {Host, Port, Opts}, []).
init({Host, Port, Opts}) ->
process_flag(trap_exit, true),
true = ets:insert_new(gun_pools, {gun_pools_key(Host, Port, Opts), self()}),
Tid = ets:new(gun_pooled_conns, [ordered_set, public]),
Size = maps:get(size, Opts, 8),
%% @todo Only start processes in static mode.
ConnOpts = conn_opts(Tid, Opts),
Conns = maps:from_list([begin
{ok, ConnPid} = gun:open(Host, Port, ConnOpts),
_ = monitor(process, ConnPid),
{ConnPid, down}
end || _ <- lists:seq(1, Size)]),
State = #state{
host=Host,
port=Port,
opts=Opts,
table=Tid,
conns=Conns
},
%% If Size is 0 then we can never be operational.
{ok, degraded, State}.
gun_pools_key(Host, Port, Opts) ->
Transport = maps:get(transport, Opts, gun:default_transport(Port)),
Authority = gun_http:host_header(Transport, Host, Port),
Scope = maps:get(scope, Opts, default),
{Scope, iolist_to_binary(Authority)}.
conn_opts(Tid, Opts) ->
ConnOpts = maps:get(conn_opts, Opts, #{}),
EventHandlerState = maps:with([event_handler], ConnOpts),
H2Opts = maps:get(http2_opts, ConnOpts, #{}),
ConnOpts#{
event_handler => {gun_pool_events_h, EventHandlerState#{
table => Tid
}},
http2_opts => H2Opts#{
notify_settings_changed => true
}
}.
%% We use the degraded state as long as at least one connection is degraded.
%% @todo Probably keep count of connections separately to avoid counting every time.
degraded(info, Msg={gun_up, ConnPid, _}, StateData=#state{opts=Opts, conns=Conns}) ->
#{ConnPid := down} = Conns,
%% We optionally run the setup function if one is defined. The
%% setup function tells us whether we are fully up or not. The
%% setup function may be called repeatedly until the connection
%% is established.
%%
%% @todo It is possible that the connection never does get
%% fully established. We should deal with this. We probably
%% need to handle all messages.
{SetupFun, SetupState0} = setup_fun(Opts),
degraded_setup(ConnPid, Msg, StateData, SetupFun, SetupState0);
%% @todo
%degraded(info, Msg={gun_tunnel_up, ConnPid, _, _}, StateData0=#state{conns=Conns}) ->
% ;
degraded(info, Msg={gun_upgrade, ConnPid, _, _, _},
StateData=#state{opts=#{setup_fun := {SetupFun, _}}, conns=Conns}) ->
%% @todo Probably shouldn't crash if the state is incorrect, that's programmer error though.
#{ConnPid := {setup, SetupState0}} = Conns,
%% We run the setup function again using the state previously kept.
degraded_setup(ConnPid, Msg, StateData, SetupFun, SetupState0);
degraded(Type, Event, StateData) ->
handle_common(Type, Event, ?FUNCTION_NAME, StateData).
setup_fun(#{setup_fun := SetupFun}) ->
SetupFun;
setup_fun(_) ->
{fun (_, {gun_up, _, Protocol}, _) ->
{up, Protocol, #{}}
end, undefined}.
degraded_setup(ConnPid, Msg, StateData0=#state{conns=Conns, conns_meta=ConnsMeta,
await_up=AwaitUp}, SetupFun, SetupState0) ->
case SetupFun(ConnPid, Msg, SetupState0) of
Setup={setup, _SetupState} ->
StateData = StateData0#state{conns=Conns#{ConnPid => Setup}},
{keep_state, StateData};
%% The Meta is different from Settings. It allows passing around
%% Websocket or tunnel stream refs.
{up, Protocol, Meta} ->
Settings = #{},
StateData = StateData0#state{
conns=Conns#{ConnPid => {up, Protocol, Settings}},
conns_meta=ConnsMeta#{ConnPid => Meta}
},
case is_degraded(StateData) of
true -> {keep_state, StateData};
false -> {next_state, operational, StateData#state{await_up=[]},
[{reply, ReplyTo, ok} || ReplyTo <- AwaitUp]}
end
end.
is_degraded(#state{conns=Conns0}) ->
Conns = maps:to_list(Conns0),
Len = length(Conns),
Ups = [up || {_, {up, _, _}} <- Conns],
Len =/= length(Ups).
operational(Type, Event, StateData) ->
handle_common(Type, Event, ?FUNCTION_NAME, StateData).
handle_common({call, From}, {checkout, _ReqOpts}, _,
StateData=#state{conns_meta=ConnsMeta}) ->
case find_available_connection(StateData) of
none ->
{keep_state_and_data, {reply, From, undefined}};
ConnPid ->
Meta = maps:get(ConnPid, ConnsMeta, #{}),
{keep_state_and_data, {reply, From, {ConnPid, Meta}}}
end;
handle_common(info, {gun_notify, ConnPid, settings_changed, Settings}, _, StateData=#state{conns=Conns}) ->
%% Assert that the state is correct.
{up, http2, _} = maps:get(ConnPid, Conns),
{keep_state, StateData#state{conns=Conns#{ConnPid => {up, http2, Settings}}}};
handle_common(info, {gun_down, ConnPid, Protocol, _Reason, _KilledStreams}, _, StateData=#state{conns=Conns}) ->
{up, Protocol, _} = maps:get(ConnPid, Conns),
{next_state, degraded, StateData#state{conns=Conns#{ConnPid => down}}};
%% @todo We do not want to reconnect automatically when the pool is dynamic.
handle_common(info, {'DOWN', _MRef, process, ConnPid0, Reason}, _,
StateData=#state{host=Host, port=Port, opts=Opts, table=Tid, conns=Conns0, conns_meta=ConnsMeta0}) ->
Conns = maps:remove(ConnPid0, Conns0),
ConnsMeta = maps:remove(ConnPid0, ConnsMeta0),
case Reason of
%% The process is down because of a configuration error.
%% Do NOT attempt to reconnect, leave the pool in a degraded state.
badarg ->
{next_state, degraded, StateData#state{conns=Conns, conns_meta=ConnsMeta}};
_ ->
ConnOpts = conn_opts(Tid, Opts),
{ok, ConnPid} = gun:open(Host, Port, ConnOpts),
_ = monitor(process, ConnPid),
{next_state, degraded, StateData#state{conns=Conns#{ConnPid => down}, conns_meta=ConnsMeta}}
end;
handle_common({call, From}, info, StateName, #state{host=Host, port=Port,
opts=Opts, table=Tid, conns=Conns, conns_meta=ConnsMeta}) ->
{keep_state_and_data, {reply, From, {StateName, #{
%% @todo Not sure whether all of this should be documented. Maybe not ConnsMeta for now?
host => Host,
port => Port,
opts => Opts,
table => Tid,
conns => Conns,
conns_meta => ConnsMeta
}}}};
handle_common({call, From}, await_up, operational, _) ->
{keep_state_and_data, {reply, From, ok}};
handle_common({call, From}, await_up, _, StateData=#state{await_up=AwaitUp}) ->
{keep_state, StateData#state{await_up=[From|AwaitUp]}};
handle_common(Type, Event, StateName, StateData) ->
logger:error("Unexpected event in state ~p of type ~p:~n~w~n~p~n",
[StateName, Type, Event, StateData]),
keep_state_and_data.
%% We go over every connection and return the first one
%% we find that has capacity. How we determine whether
%% capacity is available depends on the protocol. For
%% HTTP/2 we look into the protocol settings. The
%% current number of streams is maintained by the
%% event handler gun_pool_events_h.
find_available_connection(#state{table=Tid, conns=Conns}) ->
I = lists:sort([{rand:uniform(), K} || K <- maps:keys(Conns)]),
find_available_connection(I, Conns, Tid).
find_available_connection([], _, _) ->
none;
find_available_connection([{_, ConnPid}|I], Conns, Tid) ->
case maps:get(ConnPid, Conns) of
{up, Protocol, Settings} ->
MaxStreams = max_streams(Protocol, Settings),
CurrentStreams = case ets:lookup(Tid, ConnPid) of
[] ->
0;
[{_, CS}] ->
CS
end,
if
CurrentStreams + 1 > MaxStreams ->
find_available_connection(I, Conns, Tid);
true ->
ConnPid
end;
_ ->
find_available_connection(I, Conns, Tid)
end.
max_streams(http, _) ->
1;
max_streams(http2, #{max_concurrent_streams := MaxStreams}) ->
MaxStreams;
max_streams(http2, #{}) ->
infinity;
%% There are no streams or Gun is not aware of streams when
%% the protocol is Websocket or raw.
max_streams(ws, _) ->
infinity;
max_streams(raw, _) ->
infinity.
terminate(Reason, StateName, #state{host=Host, port=Port, opts=Opts, await_up=AwaitUp}) ->
gen_statem:reply([
{reply, ReplyTo, {error, {terminate, StateName, Reason}}}
|| ReplyTo <- AwaitUp]),
true = ets:delete(gun_pools, gun_pools_key(Host, Port, Opts)),
ok.

157
gun/src/gun_pool_events_h.erl Normal file
View File

@ -0,0 +1,157 @@
%% Copyright (c) 2021-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_pool_events_h).
-export([init/2]).
-export([domain_lookup_start/2]).
-export([domain_lookup_end/2]).
-export([connect_start/2]).
-export([connect_end/2]).
-export([tls_handshake_start/2]).
-export([tls_handshake_end/2]).
-export([request_start/2]).
-export([request_headers/2]).
-export([request_end/2]).
-export([push_promise_start/2]).
-export([push_promise_end/2]).
-export([response_start/2]).
-export([response_inform/2]).
-export([response_headers/2]).
-export([response_trailers/2]).
-export([response_end/2]).
-export([ws_upgrade/2]).
-export([ws_recv_frame_start/2]).
-export([ws_recv_frame_header/2]).
-export([ws_recv_frame_end/2]).
-export([ws_send_frame_start/2]).
-export([ws_send_frame_end/2]).
-export([protocol_changed/2]).
-export([origin_changed/2]).
-export([cancel/2]).
-export([disconnect/2]).
-export([terminate/2]).
init(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
domain_lookup_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
domain_lookup_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
connect_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
connect_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
tls_handshake_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
tls_handshake_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
request_start(Event=#{stream_ref := StreamRef}, State=#{table := Tid}) ->
_ = ets:update_counter(Tid, self(), +1, {self(), 0}),
propagate(Event, State#{
StreamRef => {nofin, nofin}
}, ?FUNCTION_NAME).
request_headers(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
request_end(Event=#{stream_ref := StreamRef}, State0=#{table := Tid}) ->
State = case State0 of
#{StreamRef := {nofin, fin}} ->
_ = ets:update_counter(Tid, self(), -1),
maps:remove(StreamRef, State0);
#{StreamRef := {nofin, IsFin}} ->
State0#{StreamRef => {fin, IsFin}}
end,
propagate(Event, State, ?FUNCTION_NAME).
push_promise_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
push_promise_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
response_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
response_inform(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
response_headers(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
response_trailers(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
response_end(Event=#{stream_ref := StreamRef}, State0=#{table := Tid}) ->
State = case State0 of
#{StreamRef := {fin, nofin}} ->
_ = ets:update_counter(Tid, self(), -1),
maps:remove(StreamRef, State0);
#{StreamRef := {IsFin, nofin}} ->
State0#{StreamRef => {IsFin, fin}}
end,
propagate(Event, State, ?FUNCTION_NAME).
ws_upgrade(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
ws_recv_frame_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
ws_recv_frame_header(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
ws_recv_frame_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
ws_send_frame_start(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
ws_send_frame_end(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
protocol_changed(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
origin_changed(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
cancel(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
disconnect(Event, State=#{table := Tid}) ->
%% The ets:delete/2 call might fail when the pool has shut down.
try
true = ets:delete(Tid, self())
catch _:_ ->
ok
end,
propagate(Event, maps:with([event_handler, table], State), ?FUNCTION_NAME).
terminate(Event, State) ->
propagate(Event, State, ?FUNCTION_NAME).
propagate(Event, State=#{event_handler := {Mod, ModState0}}, Fun) ->
ModState = Mod:Fun(Event, ModState0),
State#{event_handler => {Mod, ModState}};
propagate(_, State, _) ->
State.

37
gun/src/gun_pools_sup.erl Normal file
View File

@ -0,0 +1,37 @@
%% Copyright (c) 2021-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_pools_sup).
-behaviour(supervisor).
%% API.
-export([start_link/0]).
%% supervisor.
-export([init/1]).
%% API.
-spec start_link() -> {ok, pid()}.
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
%% supervisor.
init([]) ->
%% @todo Review restart strategies.
Procs = [
#{id => gun_pool, start => {gun_pool, start_link, []}, restart => transient}
],
{ok, {#{strategy => simple_one_for_one}, Procs}}.

72
gun/src/gun_protocols.erl Normal file
View File

@ -0,0 +1,72 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_protocols).
-export([add_stream_ref/2]).
-export([handler/1]).
-export([handler_and_opts/2]).
-export([negotiated/2]).
-export([stream_ref/1]).
-spec add_stream_ref(Protocol, undefined | gun:stream_ref())
-> Protocol when Protocol :: gun:protocol().
add_stream_ref(Protocol, undefined) ->
Protocol;
add_stream_ref({ProtocolName, ProtocolOpts}, StreamRef) ->
{ProtocolName, ProtocolOpts#{stream_ref => StreamRef}};
add_stream_ref(ProtocolName, StreamRef) ->
{ProtocolName, #{stream_ref => StreamRef}}.
-spec handler(gun:protocol()) -> module().
handler(http) -> gun_http;
handler({http, _}) -> gun_http;
handler(http2) -> gun_http2;
handler({http2, _}) -> gun_http2;
handler(raw) -> gun_raw;
handler({raw, _}) -> gun_raw;
handler(socks) -> gun_socks;
handler({socks, _}) -> gun_socks;
handler(ws) -> gun_ws;
handler({ws, _}) -> gun_ws.
-spec handler_and_opts(gun:protocol(), map()) -> {module(), map()}.
handler_and_opts({ProtocolName, ProtocolOpts}, _) ->
{handler(ProtocolName), ProtocolOpts};
handler_and_opts(ProtocolName, Opts) ->
Protocol = handler(ProtocolName),
{Protocol, maps:get(Protocol:opts_name(), Opts, #{})}.
-spec negotiated({ok, binary()} | {error, protocol_not_negotiated}, gun:protocols())
-> gun:protocol().
negotiated({ok, <<"h2">>}, Protocols) ->
lists:foldl(fun
(E = http2, _) -> E;
(E = {http2, _}, _) -> E;
(_, Acc) -> Acc
end, http2, Protocols);
negotiated({ok, <<"http/1.1">>}, Protocols) ->
lists:foldl(fun
(E = http, _) -> E;
(E = {http, _}, _) -> E;
(_, Acc) -> Acc
end, http, Protocols);
negotiated({error, protocol_not_negotiated}, [Protocol]) ->
Protocol;
negotiated({error, protocol_not_negotiated}, _) ->
http.
-spec stream_ref(gun:protocol()) -> undefined | gun:stream_ref().
stream_ref({_, ProtocolOpts}) -> maps:get(stream_ref, ProtocolOpts, undefined);
stream_ref(_) -> undefined.

9520
gun/src/gun_public_suffix.erl Normal file
View File

File diff suppressed because it is too large Load Diff

29
gun/src/gun_public_suffix.erl.src Normal file
View File

@ -0,0 +1,29 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_public_suffix).
-compile(no_type_opt). %% Until at least OTP-23.
-export([match/1]).
-spec match(binary()) -> boolean().
match(Domain) ->
Subdomains = string:split(Domain, <<".">>, all),
m(Subdomains).
%% GENERATED_M
m(_) -> false.
%% GENERATED_E
e(_) -> true.

97
gun/src/gun_raw.erl Normal file
View File

@ -0,0 +1,97 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_raw).
-export([check_options/1]).
-export([name/0]).
-export([opts_name/0]).
-export([has_keepalive/0]).
-export([init/4]).
-export([handle/5]).
-export([update_flow/4]).
-export([closing/4]).
-export([close/4]).
-export([data/7]).
-export([down/1]).
-record(raw_state, {
ref :: undefined | gun:stream_ref(),
reply_to :: pid(),
socket :: inet:socket() | ssl:sslsocket(),
transport :: module(),
flow :: integer() | infinity
}).
check_options(Opts) ->
do_check_options(maps:to_list(Opts)).
do_check_options([]) ->
ok;
do_check_options([{flow, Flow}|Opts]) when is_integer(Flow); Flow == infinity ->
do_check_options(Opts);
do_check_options([Opt|_]) ->
{error, {options, {raw, Opt}}}.
name() -> raw.
opts_name() -> raw_opts.
has_keepalive() -> false.
init(ReplyTo, Socket, Transport, Opts) ->
StreamRef = maps:get(stream_ref, Opts, undefined),
InitialFlow = maps:get(flow, Opts, infinity),
{ok, connected_data_only, #raw_state{ref=StreamRef, reply_to=ReplyTo,
socket=Socket, transport=Transport, flow=InitialFlow}}.
handle(Data, State=#raw_state{ref=StreamRef, reply_to=ReplyTo, flow=Flow0},
CookieStore, _, EvHandlerState) ->
%% When we take over the entire connection there is no stream reference.
ReplyTo ! {gun_data, self(), StreamRef, nofin, Data},
Flow = case Flow0 of
infinity -> infinity;
_ -> Flow0 - 1
end,
{[
{state, State#raw_state{flow=Flow}},
{active, Flow > 0}
], CookieStore, EvHandlerState}.
update_flow(State=#raw_state{flow=Flow0}, _ReplyTo, _StreamRef, Inc) ->
Flow = case Flow0 of
infinity -> infinity;
_ -> Flow0 + Inc
end,
[
{state, State#raw_state{flow=Flow}},
{active, Flow > 0}
].
%% We can always close immediately.
closing(_, _, _, EvHandlerState) ->
{close, EvHandlerState}.
close(_, _, _, EvHandlerState) ->
EvHandlerState.
%% @todo Initiate closing on IsFin=fin.
data(#raw_state{ref=StreamRef, socket=Socket, transport=Transport}, StreamRef,
_ReplyTo, _IsFin, Data, _EvHandler, EvHandlerState) ->
case Transport:send(Socket, Data) of
ok -> {[], EvHandlerState};
Error={error, _} -> {Error, EvHandlerState}
end.
%% raw has no concept of streams.
down(_) ->
[].

207
gun/src/gun_socks.erl Normal file
View File

@ -0,0 +1,207 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_socks).
-export([check_options/1]).
-export([name/0]).
-export([opts_name/0]).
-export([has_keepalive/0]).
-export([init/4]).
-export([switch_transport/3]).
-export([handle/5]).
-export([closing/4]).
-export([close/4]).
%% @todo down
-record(socks_state, {
ref :: undefined | gun:stream_ref(),
reply_to :: pid(),
socket :: inet:socket() | ssl:sslsocket(),
transport :: module(),
opts = #{} :: gun:socks_opts(),
%% We only support version 5 at the moment.
version :: 5,
status :: auth_method_select | auth_username_password | connect
}).
check_options(Opts=#{host := _, port := _}) ->
do_check_options(maps:to_list(maps:without([host, port], Opts)));
%% Host and port are not optional.
check_options(#{host := _}) ->
{error, options, {socks, port}};
check_options(#{}) ->
{error, options, {socks, host}}.
do_check_options([]) ->
ok;
do_check_options([Opt={auth, L}|Opts]) ->
case check_auth_opt(L) of
ok -> do_check_options(Opts);
error -> {error, {options, {socks, Opt}}}
end;
%% @todo Proper protocols check.
do_check_options([{protocols, P}|Opts]) when is_list(P) ->
do_check_options(Opts);
do_check_options([{tls_handshake_timeout, infinity}|Opts]) ->
do_check_options(Opts);
do_check_options([{tls_handshake_timeout, T}|Opts]) when is_integer(T), T >= 0 ->
do_check_options(Opts);
do_check_options([{tls_opts, L}|Opts]) when is_list(L) ->
do_check_options(Opts);
do_check_options([{transport, T}|Opts]) when T =:= tcp; T =:= tls ->
do_check_options(Opts);
do_check_options([{version, 5}|Opts]) ->
do_check_options(Opts);
do_check_options([Opt|_]) ->
{error, {options, {socks, Opt}}}.
check_auth_opt(Methods) ->
%% Methods must not appear more than once, and they
%% must be one of none or {username_password, binary(), binary()}.
Check = lists:usort([case M of
none -> ok;
{username_password, U, P} when is_binary(U), is_binary(P) -> ok
end || M <- Methods]),
case {length(Methods) =:= length(Check), lists:usort(Check)} of
{true, []} -> ok;
{true, [ok]} -> ok;
_ -> error
end.
name() -> socks.
opts_name() -> socks_opts.
has_keepalive() -> false.
init(ReplyTo, Socket, Transport, Opts) ->
StreamRef = maps:get(stream_ref, Opts, undefined),
5 = Version = maps:get(version, Opts, 5),
Auth = maps:get(auth, Opts, [none]),
Methods = <<case A of
{username_password, _, _} -> <<2>>;
none -> <<0>>
end || A <- Auth>>,
case Transport:send(Socket, [<<5, (length(Auth))>>, Methods]) of
ok ->
{ok, connected_no_input, #socks_state{ref=StreamRef, reply_to=ReplyTo,
socket=Socket, transport=Transport,
opts=Opts, version=Version, status=auth_method_select}};
Error={error, _Reason} ->
Error
end.
switch_transport(Transport, Socket, State) ->
State#socks_state{socket=Socket, transport=Transport}.
handle(Data, State, CookieStore, _, EvHandlerState) ->
{handle(Data, State), CookieStore, EvHandlerState}.
%% No authentication.
handle(<<5, 0>>, State=#socks_state{version=5, status=auth_method_select}) ->
case send_socks5_connect(State) of
ok -> {state, State#socks_state{status=connect}};
Error={error, _} -> Error
end;
%% Username/password authentication.
handle(<<5, 2>>, State=#socks_state{socket=Socket, transport=Transport, opts=#{auth := AuthMethods},
version=5, status=auth_method_select}) ->
[{username_password, Username, Password}] = [Method || Method <- AuthMethods],
ULen = byte_size(Username),
PLen = byte_size(Password),
case Transport:send(Socket, <<1, ULen, Username/binary, PLen, Password/binary>>) of
ok -> {state, State#socks_state{status=auth_username_password}};
Error={error, _} -> Error
end;
%% Username/password authentication successful.
handle(<<1, 0>>, State=#socks_state{version=5, status=auth_username_password}) ->
case send_socks5_connect(State) of
ok -> {state, State#socks_state{status=connect}};
Error={error, _} -> Error
end;
%% Username/password authentication error.
handle(<<1, _>>, #socks_state{version=5, status=auth_username_password}) ->
{error, {socks5, username_password_auth_failure}};
%% Connect reply.
handle(<<5, 0, 0, Rest0/bits>>, #socks_state{ref=StreamRef, reply_to=ReplyTo, opts=Opts,
version=5, status=connect}) ->
%% @todo What to do with BoundAddr and BoundPort? Add as metadata to origin info?
{_BoundAddr, _BoundPort} = case Rest0 of
%% @todo Seen a server with <<1, 0:48>>.
<<1, A, B, C, D, Port:16>> ->
{{A, B, C, D}, Port};
<<3, Len, Host:Len/binary, Port:16>> ->
%% We convert to list to get an inet:hostname().
{unicode:characters_to_list(Host), Port};
<<4, A:16, B:16, C:16, D:16, E:16, F:16, G:16, H:16, Port:16>> ->
{{A, B, C, D, E, F, G, H}, Port}
end,
%% @todo Maybe an event indicating success.
#{host := NewHost, port := NewPort} = Opts,
%% There is no origin scheme when not using HTTP but we act as if
%% there is and simply correct the value in the info functions.
case Opts of
#{transport := tls} ->
HandshakeEvent0 = #{
reply_to => ReplyTo,
tls_opts => maps:get(tls_opts, Opts, []),
timeout => maps:get(tls_handshake_timeout, Opts, infinity)
},
HandshakeEvent = case StreamRef of
undefined -> HandshakeEvent0;
_ -> HandshakeEvent0#{stream_ref => StreamRef}
end,
[{origin, <<"https">>, NewHost, NewPort, socks5},
{tls_handshake, HandshakeEvent, maps:get(protocols, Opts, [http2, http]), ReplyTo}];
_ ->
[NewProtocol0] = maps:get(protocols, Opts, [http]),
NewProtocol = gun_protocols:add_stream_ref(NewProtocol0, StreamRef),
Protocol = gun_protocols:handler(NewProtocol),
ReplyTo ! {gun_tunnel_up, self(), StreamRef, Protocol:name()},
[{origin, <<"http">>, NewHost, NewPort, socks5},
{switch_protocol, NewProtocol, ReplyTo}]
end;
handle(<<5, Error, _/bits>>, #socks_state{version=5, status=connect}) ->
Reason = case Error of
1 -> general_socks_server_failure;
2 -> connection_not_allowed_by_ruleset;
3 -> network_unreachable;
4 -> host_unreachable;
5 -> connection_refused;
6 -> ttl_expired;
7 -> command_not_supported;
8 -> address_type_not_supported;
_ -> {unknown_error, Error}
end,
{error, {socks5, Reason}}.
send_socks5_connect(#socks_state{socket=Socket, transport=Transport, opts=Opts}) ->
ATypeAndDestAddr = case maps:get(host, Opts) of
{A, B, C, D} -> <<1, A, B, C, D>>;
{A, B, C, D, E, F, G, H} -> <<4, A:16, B:16, C:16, D:16, E:16, F:16, G:16, H:16>>;
Host when is_atom(Host) ->
DestAddr0 = atom_to_binary(Host, utf8),
<<3, (byte_size(DestAddr0)), DestAddr0/binary>>;
Host ->
DestAddr0 = unicode:characters_to_binary(Host, utf8),
<<3, (byte_size(DestAddr0)), DestAddr0/binary>>
end,
DestPort = maps:get(port, Opts),
Transport:send(Socket, <<5, 1, 0, ATypeAndDestAddr/binary, DestPort:16>>).
%% We can always close immediately.
closing(_, _, _, EvHandlerState) ->
{close, EvHandlerState}.
close(_, _, _, EvHandlerState) ->
EvHandlerState.

63
gun/src/gun_sse_h.erl Normal file
View File

@ -0,0 +1,63 @@
%% Copyright (c) 2017-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_sse_h).
-behavior(gun_content_handler).
-export([init/5]).
-export([handle/3]).
-record(state, {
reply_to :: pid(),
stream_ref :: reference(),
sse_state :: cow_sse:state()
}).
%% @todo In the future we want to allow different media types.
-spec init(pid(), reference(), _, cow_http:headers(), _)
-> {ok, #state{}} | disable.
init(ReplyTo, StreamRef, _, Headers, _) ->
case lists:keyfind(<<"content-type">>, 1, Headers) of
{_, ContentType} ->
case cow_http_hd:parse_content_type(ContentType) of
{<<"text">>, <<"event-stream">>, _Ignored} ->
{ok, #state{reply_to=ReplyTo, stream_ref=StreamRef,
sse_state=cow_sse:init()}};
_ ->
disable
end;
_ ->
disable
end.
-spec handle(_, binary(), State) -> {done, non_neg_integer(), State} when State::#state{}.
handle(IsFin, Data, State) ->
handle(IsFin, Data, State, 0).
handle(IsFin, Data, State=#state{reply_to=ReplyTo, stream_ref=StreamRef, sse_state=SSE0}, Flow) ->
case cow_sse:parse(Data, SSE0) of
{event, Event, SSE} ->
ReplyTo ! {gun_sse, self(), StreamRef, Event},
handle(IsFin, <<>>, State#state{sse_state=SSE}, Flow + 1);
{more, SSE} ->
Inc = case IsFin of
fin ->
ReplyTo ! {gun_sse, self(), StreamRef, fin},
1;
_ ->
0
end,
{done, Flow + Inc, State#state{sse_state=SSE}}
end.

37
gun/src/gun_sup.erl Normal file
View File

@ -0,0 +1,37 @@
%% Copyright (c) 2013-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_sup).
-behaviour(supervisor).
%% API.
-export([start_link/0]).
%% supervisor.
-export([init/1]).
%% API.
-spec start_link() -> {ok, pid()}.
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
%% supervisor.
init([]) ->
Procs = [
#{id => gun_conns_sup, start => {gun_conns_sup, start_link, []}, type => supervisor},
#{id => gun_pools_sup, start => {gun_pools_sup, start_link, []}, type => supervisor}
],
{ok, {#{}, Procs}}.

112
gun/src/gun_tcp.erl Normal file
View File

@ -0,0 +1,112 @@
%% Copyright (c) 2011-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_tcp).
-export([name/0]).
-export([messages/0]).
-export([domain_lookup/4]).
-export([connect/2]).
-export([send/2]).
-export([setopts/2]).
-export([sockname/1]).
-export([close/1]).
-type lookup_info() :: #{
ip_addresses := [inet:ip_address()],
port := inet:port_number(),
tcp_module := module(),
tcp_opts := [gen_tcp:connect_option()]
}.
-export_type([lookup_info/0]).
name() -> tcp.
messages() -> {tcp, tcp_closed, tcp_error}.
%% The functions domain_lookup/4 and connect/2 are very similar
%% to gen_tcp:connect/4 except the logic is split in order to
%% be able to trigger events between the domain lookup step
%% and the actual connect step.
-spec domain_lookup(inet:ip_address() | inet:hostname(),
inet:port_number(), [gen_tcp:connect_option()], timeout())
-> {ok, lookup_info()} | {error, atom()}.
domain_lookup(Address, Port0, Opts0, Timeout) ->
{Mod, Opts} = inet:tcp_module(Opts0, Address),
Timer = inet:start_timer(Timeout),
try Mod:getaddrs(Address, Timer) of
{ok, IPs} ->
case Mod:getserv(Port0) of
{ok, Port} ->
{ok, #{
ip_addresses => IPs,
port => Port,
tcp_module => Mod,
tcp_opts => Opts ++ [binary, {active, false}, {packet, raw}]
}};
Error ->
maybe_exit(Error)
end;
Error ->
maybe_exit(Error)
after
_ = inet:stop_timer(Timer)
end.
-spec connect(lookup_info(), timeout())
-> {ok, inet:socket()} | {error, atom()}.
connect(#{ip_addresses := IPs, port := Port, tcp_module := Mod, tcp_opts := Opts}, Timeout) ->
Timer = inet:start_timer(Timeout),
Res = try
try_connect(IPs, Port, Opts, Timer, Mod, {error, einval})
after
_ = inet:stop_timer(Timer)
end,
case Res of
{ok, S} -> {ok, S};
Error -> maybe_exit(Error)
end.
try_connect([IP|IPs], Port, Opts, Timer, Mod, _) ->
Timeout = inet:timeout(Timer),
case Mod:connect(IP, Port, Opts, Timeout) of
{ok, S} -> {ok, S};
{error, einval} -> {error, einval};
{error, timeout} -> {error, timeout};
Error -> try_connect(IPs, Port, Opts, Timer, Mod, Error)
end;
try_connect([], _, _, _, _, Error) ->
Error.
maybe_exit({error, einval}) -> exit(badarg);
maybe_exit({error, eaddrnotavail}) -> exit(badarg);
maybe_exit(Error) -> Error.
-spec send(inet:socket(), iodata()) -> ok | {error, atom()}.
send(Socket, Packet) ->
gen_tcp:send(Socket, Packet).
-spec setopts(inet:socket(), list()) -> ok | {error, atom()}.
setopts(Socket, Opts) ->
inet:setopts(Socket, Opts).
-spec sockname(inet:socket())
-> {ok, {inet:ip_address(), inet:port_number()}} | {error, atom()}.
sockname(Socket) ->
inet:sockname(Socket).
-spec close(inet:socket()) -> ok.
close(Socket) ->
gen_tcp:close(Socket).

70
gun/src/gun_tcp_proxy.erl Normal file
View File

@ -0,0 +1,70 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_tcp_proxy).
-export([name/0]).
-export([messages/0]).
-export([connect/3]).
-export([connect/4]).
-export([send/2]).
-export([setopts/2]).
-export([sockname/1]).
-export([close/1]).
-type socket() :: #{
%% The pid of the Gun connection.
gun_pid := pid(),
%% The pid of the process that gets replies for this tunnel.
reply_to := pid(),
%% The full stream reference for this tunnel.
stream_ref := gun:stream_ref()
}.
name() -> tcp_proxy.
messages() -> {tcp_proxy, tcp_proxy_closed, tcp_proxy_error}.
-spec connect(_, _, _) -> no_return().
connect(_, _, _) ->
error(not_implemented).
-spec connect(_, _, _, _) -> no_return().
connect(_, _, _, _) ->
error(not_implemented).
-spec send(socket(), iodata()) -> ok.
send(#{gun_pid := GunPid, reply_to := ReplyTo, stream_ref := StreamRef,
handle_continue_stream_ref := ContinueStreamRef}, Data) ->
GunPid ! {handle_continue, ContinueStreamRef, {data, ReplyTo, StreamRef, nofin, Data}},
ok;
send(#{reply_to := ReplyTo, stream_ref := StreamRef}, Data) ->
gen_statem:cast(self(), {data, ReplyTo, StreamRef, nofin, Data}).
-spec setopts(_, _) -> no_return().
setopts(#{handle_continue_stream_ref := _}, _) ->
%% We send messages automatically regardless of active mode.
ok;
setopts(_, _) ->
error(not_implemented).
-spec sockname(_) -> no_return().
sockname(_) ->
error(not_implemented).
-spec close(socket()) -> ok.
close(_) ->
ok.

49
gun/src/gun_tls.erl Normal file
View File

@ -0,0 +1,49 @@
%% Copyright (c) 2011-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_tls).
-export([name/0]).
-export([messages/0]).
-export([connect/3]).
-export([send/2]).
-export([setopts/2]).
-export([sockname/1]).
-export([close/1]).
name() -> tls.
messages() -> {ssl, ssl_closed, ssl_error}.
-spec connect(inet:socket(), any(), timeout())
-> {ok, ssl:sslsocket()} | {error, atom()}.
connect(Socket, Opts, Timeout) ->
ssl:connect(Socket, Opts, Timeout).
-spec send(ssl:sslsocket(), iodata()) -> ok | {error, atom()}.
send(Socket, Packet) ->
ssl:send(Socket, Packet).
-spec setopts(ssl:sslsocket(), list()) -> ok | {error, atom()}.
setopts(Socket, Opts) ->
ssl:setopts(Socket, Opts).
-spec sockname(ssl:sslsocket())
-> {ok, {inet:ip_address(), inet:port_number()}} | {error, atom()}.
sockname(Socket) ->
ssl:sockname(Socket).
-spec close(ssl:sslsocket()) -> ok.
close(Socket) ->
ssl:close(Socket).

492
gun/src/gun_tls_proxy.erl Normal file
View File

@ -0,0 +1,492 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
%% Intermediary process for proxying TLS connections. This process
%% is started by Gun when CONNECT request is issued and stays alive
%% while the proxy connection exists.
%%
%% Data comes in and out of this process, which is responsible for
%% passing incoming/outgoing data through the fake ssl socket process
%% it created to perform the decoding/encoding operations.
%%
%% Normal scenario:
%% Gun process -> TLS socket
%%
%% One proxy socket scenario:
%% Gun process -> gun_tls_proxy (proxied socket) -> TLS socket
%%
%% N proxy socket scenario:
%% Gun process -> gun_tls_proxy -> ... -> gun_tls_proxy -> TLS socket
%%
%% The difficult part is the connection. Because ssl:connect/4 does
%% not return until the connection is setup, and we need to send and
%% receive data for the TLS handshake, we need a temporary process
%% to call this function, and communicate with it. Once the connection
%% is setup the temporary process is gone and things go back to normal.
%% The send operations also require a temporary process in order to avoid
%% blocking because the same gun_tls_proxy process must send twice (first
%% to the fake ssl socket and then to the outgoing socket).
-module(gun_tls_proxy).
-behaviour(gen_statem).
%% Gun-specific interface.
-export([start_link/7]).
%% gun_tls_proxy_cb interface.
-export([cb_controlling_process/2]).
-export([cb_send/2]).
-export([cb_setopts/2]).
%% Gun transport.
-export([name/0]).
-export([messages/0]).
-export([connect/3]).
-export([connect/4]).
-export([send/2]).
-export([setopts/2]).
-export([sockname/1]).
-export([close/1]).
%% Internals.
-export([callback_mode/0]).
-export([init/1]).
-export([connect_proc/5]).
-export([not_connected/3]).
-export([connected/3]).
-record(state, {
%% The pid of the owner process. This is where we send active messages.
owner_pid :: pid(),
owner_active = false :: false | once | true | pos_integer(),
owner_buffer = <<>> :: binary(),
%% The host/port the fake ssl socket thinks it's connected to.
host :: inet:ip_address() | inet:hostname(),
port :: inet:port_number(),
%% The fake ssl socket we are using in the proxy.
proxy_socket :: any(),
proxy_pid :: pid(),
proxy_active = false :: false | once | true | pos_integer(),
proxy_buffer = <<>> :: binary(),
%% The socket or proxy process we are sending to.
out_socket :: any(),
out_transport :: module(),
out_messages :: {atom(), atom(), atom()}, %% @todo Missing passive.
%% Extra information to be sent to the owner when the handshake completes.
extra :: any()
}).
%-define(DEBUG_PROXY,1).
-ifdef(DEBUG_PROXY).
-define(DEBUG_LOG(Format, Args),
io:format(user, "(~p) ~p:~p/~p:" ++ Format ++ "~n",
[self(), ?MODULE, ?FUNCTION_NAME, ?FUNCTION_ARITY] ++ Args)).
-else.
-define(DEBUG_LOG(Format, Args), _ = Format, _ = Args, ok).
-endif.
%% Gun-specific interface.
start_link(Host, Port, Opts, Timeout, OutSocket, OutTransport, Extra) ->
?DEBUG_LOG("host ~0p port ~0p opts ~0p timeout ~0p out_socket ~0p out_transport ~0p",
[Host, Port, Opts, Timeout, OutSocket, OutTransport]),
case gen_statem:start_link(?MODULE,
{self(), Host, Port, Opts, Timeout, OutSocket, OutTransport, Extra},
[]) of
{ok, Pid} when is_port(OutSocket) ->
ok = gen_tcp:controlling_process(OutSocket, Pid),
{ok, Pid};
{ok, Pid} when is_map(OutSocket) ->
{ok, Pid};
{ok, Pid} when not is_pid(OutSocket) ->
ok = ssl:controlling_process(OutSocket, Pid),
{ok, Pid};
Other ->
Other
end.
%% gun_tls_proxy_cb interface.
cb_controlling_process(Pid, ControllingPid) ->
?DEBUG_LOG("pid ~0p controlling_pid ~0p", [Pid, ControllingPid]),
gen_statem:cast(Pid, {?FUNCTION_NAME, ControllingPid}).
cb_send(Pid, Data) ->
?DEBUG_LOG("pid ~0p data ~0p", [Pid, Data]),
try
gen_statem:call(Pid, {?FUNCTION_NAME, Data})
catch
exit:{noproc, _} ->
{error, closed}
end.
cb_setopts(Pid, Opts) ->
?DEBUG_LOG("pid ~0p opts ~0p", [Pid, Opts]),
try
gen_statem:call(Pid, {?FUNCTION_NAME, Opts})
catch
exit:{noproc, _} ->
{error, einval}
end.
%% Transport.
name() -> tls_proxy.
messages() -> {tls_proxy, tls_proxy_closed, tls_proxy_error}.
-spec connect(_, _, _) -> no_return().
connect(_, _, _) ->
error(not_implemented).
-spec connect(_, _, _, _) -> no_return().
connect(_, _, _, _) ->
error(not_implemented).
-spec send(pid(), iodata()) -> ok | {error, atom()}.
send(Pid, Data) ->
?DEBUG_LOG("pid ~0p data ~0p", [Pid, Data]),
gen_statem:call(Pid, {?FUNCTION_NAME, Data}).
-spec setopts(pid(), list()) -> ok.
setopts(Pid, Opts) ->
?DEBUG_LOG("pid ~0p opts ~0p", [Pid, Opts]),
gen_statem:cast(Pid, {?FUNCTION_NAME, Opts}).
-spec sockname(pid())
-> {ok, {inet:ip_address(), inet:port_number()}} | {error, atom()}.
sockname(Pid) ->
?DEBUG_LOG("pid ~0p", [Pid]),
gen_statem:call(Pid, ?FUNCTION_NAME).
-spec close(pid()) -> ok.
close(Pid) ->
?DEBUG_LOG("pid ~0p", [Pid]),
try
gen_statem:call(Pid, ?FUNCTION_NAME)
catch
%% May happen for example when the handshake fails.
exit:{noproc, _} ->
ok
end.
%% gen_statem.
callback_mode() -> state_functions.
init({OwnerPid, Host, Port, Opts, Timeout, OutSocket, OutTransport, Extra}) ->
if
is_pid(OutSocket) ->
gen_statem:cast(OutSocket, {set_owner, self()});
true ->
ok
end,
Messages = case OutTransport of
gen_tcp -> {tcp, tcp_closed, tcp_error};
ssl -> {ssl, ssl_closed, ssl_error};
_ -> OutTransport:messages()
end,
ProxyPid = spawn_link(?MODULE, connect_proc, [self(), Host, Port, Opts, Timeout]),
?DEBUG_LOG("owner_pid ~0p host ~0p port ~0p opts ~0p timeout ~0p"
" out_socket ~0p out_transport ~0p proxy_pid ~0p",
[OwnerPid, Host, Port, Opts, Timeout, OutSocket, OutTransport, ProxyPid]),
{ok, not_connected, #state{owner_pid=OwnerPid, host=Host, port=Port, proxy_pid=ProxyPid,
out_socket=OutSocket, out_transport=OutTransport, out_messages=Messages,
extra=Extra}}.
connect_proc(ProxyPid, Host, Port, Opts, Timeout) ->
?DEBUG_LOG("proxy_pid ~0p host ~0p port ~0p opts ~0p timeout ~0p",
[ProxyPid, Host, Port, Opts, Timeout]),
_ = case ssl:connect(Host, Port, [
{active, false}, binary,
{cb_info, {gun_tls_proxy_cb, tls_proxy, tls_proxy_closed, tls_proxy_error}},
{?MODULE, ProxyPid}
|Opts], Timeout) of
{ok, Socket} ->
?DEBUG_LOG("socket ~0p", [Socket]),
ok = ssl:controlling_process(Socket, ProxyPid),
gen_statem:cast(ProxyPid, {?FUNCTION_NAME, {ok, Socket}});
Error ->
?DEBUG_LOG("error ~0p", [Error]),
gen_statem:cast(ProxyPid, {?FUNCTION_NAME, Error})
end,
ok.
%% Postpone events that require the proxy socket to be up.
not_connected({call, _}, Msg={send, _}, State) ->
?DEBUG_LOG("postpone ~0p state ~0p", [Msg, State]),
{keep_state_and_data, postpone};
not_connected(cast, Msg={setopts, _}, State) ->
?DEBUG_LOG("postpone ~0p state ~0p", [Msg, State]),
{keep_state_and_data, postpone};
not_connected(cast, Msg={connect_proc, {ok, Socket}}, State=#state{owner_pid=OwnerPid, extra=Extra}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {?MODULE, self(), {ok, ssl:negotiated_protocol(Socket)}, Extra},
%% We need to spawn this call before OTP-21.2 because it triggers
%% a cb_setopts call that blocks us. Might be OK to just leave it
%% like this once we support 21.2+ only.
spawn(fun() -> ok = ssl:setopts(Socket, [{active, true}]) end),
{next_state, connected, State#state{proxy_socket=Socket}};
not_connected(cast, Msg={connect_proc, Error}, State=#state{owner_pid=OwnerPid, extra=Extra}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {?MODULE, self(), Error, Extra},
%% We unlink from the owner process to avoid taking it down with us.
unlink(OwnerPid),
{stop, {shutdown, Error}, State};
not_connected(Type, Event, State) ->
handle_common(Type, Event, State).
%% Send data through the proxy socket.
connected({call, From}, Msg={send, Data}, State=#state{proxy_socket=Socket}) ->
?DEBUG_LOG("msg ~0p from ~0p state ~0p", [Msg, From, State]),
Self = self(),
SpawnedPid = spawn(fun() ->
gen_statem:cast(Self, {send_result, From, ssl:send(Socket, Data)})
end),
?DEBUG_LOG("spawned ~0p", [SpawnedPid]),
keep_state_and_data;
%% Messages from the proxy socket.
%%
%% When the out_socket is a #{stream_ref := _} map we know that processing
%% of the data isn't yet complete. We wrap the message in a handle_continue
%% tuple and provide the StreamRef for further processing.
connected(info, Msg={ssl, Socket, Data}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket,
out_socket=#{handle_continue_stream_ref := StreamRef}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {handle_continue, StreamRef, {tls_proxy, self(), Data}},
keep_state_and_data;
connected(info, Msg={ssl_closed, Socket}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket,
out_socket=#{handle_continue_stream_ref := StreamRef}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {handle_continue, StreamRef, {tls_proxy_closed, self()}},
keep_state_and_data;
connected(info, Msg={ssl_error, Socket, Reason}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket,
out_socket=#{handle_continue_stream_ref := StreamRef}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {handle_continue, StreamRef, {tls_proxy_error, self(), Reason}},
keep_state_and_data;
%% When the out_socket is anything else then the data is sent like normal
%% socket data. It does not need to be handled specially.
connected(info, Msg={ssl, Socket, Data}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {tls_proxy, self(), Data},
keep_state_and_data;
connected(info, Msg={ssl_closed, Socket}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {tls_proxy_closed, self()},
keep_state_and_data;
connected(info, Msg={ssl_error, Socket, Reason}, State=#state{owner_pid=OwnerPid, proxy_socket=Socket}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
OwnerPid ! {tls_proxy_error, self(), Reason},
keep_state_and_data;
connected(Type, Event, State) ->
handle_common(Type, Event, State).
handle_common({call, From}, Msg={cb_send, Data}, State=#state{
out_socket=OutSocket, out_transport=OutTransport}) ->
?DEBUG_LOG("msg ~0p from ~0p state ~0p", [Msg, From, State]),
Self = self(),
SpawnedPid = spawn(fun() ->
gen_statem:cast(Self, {send_result, From, OutTransport:send(OutSocket, Data)})
end),
?DEBUG_LOG("spawned ~0p", [SpawnedPid]),
keep_state_and_data;
handle_common({call, From}, Msg={cb_setopts, Opts}, State=#state{
out_socket=OutSocket, out_transport=OutTransport0}) ->
?DEBUG_LOG("msg ~0p from ~0p state ~0p", [Msg, From, State]),
OutTransport = case OutTransport0 of
gen_tcp -> inet;
_ -> OutTransport0
end,
{keep_state, proxy_setopts(Opts, State),
{reply, From, OutTransport:setopts(OutSocket, [{active, true}])}};
handle_common({call, From}, Msg=sockname, State=#state{
out_socket=OutSocket, out_transport=OutTransport}) ->
?DEBUG_LOG("msg ~0p from ~0p state ~0p", [Msg, From, State]),
{keep_state, State,
{reply, From, OutTransport:sockname(OutSocket)}};
handle_common({call, From}, Msg=close, State) ->
?DEBUG_LOG("msg ~0p from ~0p state ~0p", [Msg, From, State]),
{stop_and_reply, {shutdown, close}, {reply, From, ok}};
handle_common(cast, Msg={set_owner, OwnerPid}, State) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
{keep_state, State#state{owner_pid=OwnerPid}};
handle_common(cast, Msg={cb_controlling_process, ProxyPid}, State) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
{keep_state, State#state{proxy_pid=ProxyPid}};
handle_common(cast, Msg={setopts, Opts}, State) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
{keep_state, owner_setopts(Opts, State)};
handle_common(cast, Msg={send_result, From, Result}, State) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
gen_statem:reply(From, Result),
keep_state_and_data;
%% Messages from the real socket.
%% @todo Make _Socket and __Socket match again.
handle_common(info, Msg={OK, _Socket, Data}, State=#state{proxy_pid=ProxyPid,
out_socket=__Socket, out_messages={OK, _, _}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
ProxyPid ! {tls_proxy, self(), Data},
keep_state_and_data;
handle_common(info, Msg={Closed, Socket}, State=#state{proxy_pid=ProxyPid,
out_socket=Socket, out_messages={_, Closed, _}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
ProxyPid ! {tls_proxy_closed, self()},
keep_state_and_data;
handle_common(info, Msg={Error, Socket, Reason}, State=#state{proxy_pid=ProxyPid,
out_socket=Socket, out_messages={_, _, Error}}) ->
?DEBUG_LOG("msg ~0p state ~0p", [Msg, State]),
ProxyPid ! {tls_proxy_error, self(), Reason},
keep_state_and_data;
%% Other messages.
handle_common(Type, Msg, State) ->
?DEBUG_LOG("IGNORED type ~0p msg ~0p state ~0p", [Type, Msg, State]),
case Type of
{call, From} ->
{keep_state, State, {reply, From, {error, bad_call}}};
_ ->
keep_state_and_data
end.
%% Internal.
owner_setopts(Opts, State0) ->
case [A || {active, A} <- Opts] of
[] -> State0;
[false] -> State0#state{owner_active=false};
% [0] -> OwnerPid ! {tls_proxy_passive, self()}, State0#state{owner_active=false};
[Active] -> owner_active(State0#state{owner_active=Active})
end.
owner_active(State=#state{owner_buffer= <<>>}) ->
State;
owner_active(State=#state{owner_active=false}) ->
State;
owner_active(State=#state{owner_pid=OwnerPid, owner_active=Active0, owner_buffer=Buffer}) ->
OwnerPid ! {tls_proxy, self(), Buffer},
Active = case Active0 of
true -> true;
once -> false%;
% 1 -> OwnerPid ! {tls_proxy_passive, self()}, false;
% N -> N - 1
end,
State#state{owner_active=Active, owner_buffer= <<>>}.
proxy_setopts(Opts, State0=#state{proxy_socket=ProxySocket, proxy_pid=ProxyPid}) ->
case [A || {active, A} <- Opts] of
[] -> State0;
[false] -> State0#state{proxy_active=false};
[0] -> ProxyPid ! {tls_proxy_passive, ProxySocket}, State0#state{proxy_active=false};
[Active] -> proxy_active(State0#state{proxy_active=Active})
end.
proxy_active(State=#state{proxy_buffer= <<>>}) ->
State;
proxy_active(State=#state{proxy_active=false}) ->
State;
proxy_active(State=#state{proxy_pid=ProxyPid, proxy_active=Active0, proxy_buffer=Buffer}) ->
ProxyPid ! {tls_proxy, self(), Buffer},
Active = case Active0 of
true -> true;
once -> false;
%% Note that tcp_passive is hardcoded in ssl until OTP-21.3.1.
1 -> ProxyPid ! {tcp_passive, self()}, false;
N -> N - 1
end,
State#state{proxy_active=Active, proxy_buffer= <<>>}.
-ifdef(DISABLED_TEST).
proxy_test_() ->
{timeout, 15000, [
{"TCP proxy", fun proxy_tcp1_t/0},
{"TLS proxy", fun proxy_ssl1_t/0},
{"Double TLS proxy", fun proxy_ssl2_t/0}
]}.
proxy_tcp1_t() ->
ssl:start(),
{ok, Socket} = gen_tcp:connect("google.com", 443, [binary, {active, false}]),
{ok, ProxyPid1} = start_link("google.com", 443, [], 5000, Socket, gen_tcp, #{}),
send(ProxyPid1, <<"GET / HTTP/1.1\r\nHost: google.com\r\n\r\n">>),
receive {tls_proxy, ProxyPid1, <<"HTTP/1.1 ", _/bits>>} -> ok end.
proxy_ssl1_t() ->
ssl:start(),
_ = (catch ct_helper:make_certs_in_ets()),
{ok, _, Port} = do_proxy_start("google.com", 443),
{ok, Socket} = ssl:connect("localhost", Port, [binary, {active, false}]),
{ok, ProxyPid1} = start_link("google.com", 443, [], 5000, Socket, ssl, #{}),
send(ProxyPid1, <<"GET / HTTP/1.1\r\nHost: google.com\r\n\r\n">>),
receive {tls_proxy, ProxyPid1, <<"HTTP/1.1 ", _/bits>>} -> ok end.
proxy_ssl2_t() ->
ssl:start(),
_ = (catch ct_helper:make_certs_in_ets()),
{ok, _, Port1} = do_proxy_start("google.com", 443),
{ok, _, Port2} = do_proxy_start("localhost", Port1),
{ok, Socket} = ssl:connect("localhost", Port2, [binary, {active, false}]),
{ok, ProxyPid1} = start_link("localhost", Port1, [], 5000, Socket, ssl, #{}),
{ok, ProxyPid2} = start_link("google.com", 443, [], 5000, ProxyPid1, ?MODULE, #{}),
send(ProxyPid2, <<"GET / HTTP/1.1\r\nHost: google.com\r\n\r\n">>),
receive {tls_proxy, ProxyPid2, <<"HTTP/1.1 ", _/bits>>} -> ok end.
do_proxy_start(Host, Port) ->
Self = self(),
Pid = spawn_link(fun() -> do_proxy_init(Self, Host, Port) end),
ListenPort = receive_from(Pid),
{ok, Pid, ListenPort}.
do_proxy_init(Parent, Host, Port) ->
Opts = ct_helper:get_certs_from_ets(),
{ok, ListenSocket} = ssl:listen(0, [binary, {active, false}|Opts]),
{ok, {_, ListenPort}} = ssl:sockname(ListenSocket),
Parent ! {self(), ListenPort},
{ok, ClientSocket0} = ssl:transport_accept(ListenSocket, 10000),
{ok, ClientSocket} = ssl:handshake(ClientSocket0, 10000),
{ok, OriginSocket} = gen_tcp:connect(
Host, Port,
[binary, {active, false}]),
ssl:setopts(ClientSocket, [{active, true}]),
inet:setopts(OriginSocket, [{active, true}]),
do_proxy_loop(ClientSocket, OriginSocket).
do_proxy_loop(ClientSocket, OriginSocket) ->
receive
{ssl, ClientSocket, Data} ->
ok = gen_tcp:send(OriginSocket, Data),
do_proxy_loop(ClientSocket, OriginSocket);
{tcp, OriginSocket, Data} ->
ok = ssl:send(ClientSocket, Data),
do_proxy_loop(ClientSocket, OriginSocket);
{tcp_closed, _} ->
ok;
Msg ->
error(Msg)
end.
receive_from(Pid) ->
receive_from(Pid, 5000).
receive_from(Pid, Timeout) ->
receive
{Pid, Msg} ->
Msg
after Timeout ->
error(timeout)
end.
-endif.

41
gun/src/gun_tls_proxy_cb.erl Normal file
View File

@ -0,0 +1,41 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
%% Transport callback for ssl.
-module(gun_tls_proxy_cb).
-export([connect/4]).
-export([controlling_process/2]).
-export([send/2]).
-export([setopts/2]).
-export([close/1]).
%% The connect/4 function is called by the process
%% that calls ssl:connect/2,3,4.
connect(_Address, _Port, Opts, _Timeout) ->
{_, GunPid} = lists:keyfind(gun_tls_proxy, 1, Opts),
{ok, GunPid}.
controlling_process(Socket, ControllingPid) ->
gun_tls_proxy:cb_controlling_process(Socket, ControllingPid).
send(Socket, Data) ->
gun_tls_proxy:cb_send(Socket, Data).
setopts(Socket, Opts) ->
gun_tls_proxy:cb_setopts(Socket, Opts).
close(_) ->
ok.

69
gun/src/gun_tls_proxy_http2_connect.erl Normal file
View File

@ -0,0 +1,69 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_tls_proxy_http2_connect).
-export([name/0]).
-export([messages/0]).
-export([connect/3]).
-export([connect/4]).
-export([send/2]).
-export([setopts/2]).
-export([sockname/1]).
-export([close/1]).
-type socket() :: #{
%% The pid of the Gun connection.
gun_pid := pid(),
%% The pid of the process that gets replies for this tunnel.
reply_to := pid(),
%% The full stream reference for this tunnel.
stream_ref := gun:stream_ref(),
%% The full stream reference for the responsible HTTP/2 stream.
handle_continue_stream_ref := gun:stream_ref()
}.
name() -> tls_proxy_http2_connect.
messages() -> {tls_proxy_http2_connect, tls_proxy_http2_connect_closed, tls_proxy_http2_connect_error}.
-spec connect(_, _, _) -> no_return().
connect(_, _, _) ->
error(not_implemented).
-spec connect(_, _, _, _) -> no_return().
connect(_, _, _, _) ->
error(not_implemented).
-spec send(socket(), iodata()) -> ok.
send(#{gun_pid := GunPid, reply_to := ReplyTo, stream_ref := DataStreamRef,
handle_continue_stream_ref := StreamRef}, Data) ->
GunPid ! {handle_continue, StreamRef, {data, ReplyTo, DataStreamRef, nofin, Data}},
ok.
-spec setopts(_, _) -> no_return().
setopts(_, _) ->
%% We send messages automatically regardless of active mode.
ok.
-spec sockname(_) -> no_return().
sockname(_) ->
error(not_implemented).
-spec close(socket()) -> ok.
close(_) ->
ok.

637
gun/src/gun_tunnel.erl Normal file
View File

@ -0,0 +1,637 @@
%% Copyright (c) 2020-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
%% This module is used when a tunnel is established and either
%% StreamRef dereference or a TLS proxy process must be handled
%% by the tunnel layer.
-module(gun_tunnel).
-export([init/6]).
-export([handle/5]).
-export([handle_continue/6]).
-export([update_flow/4]).
-export([closing/4]).
-export([close/4]).
-export([keepalive/3]).
-export([headers/12]).
-export([request/13]).
-export([data/7]).
-export([connect/9]).
-export([cancel/5]).
-export([timeout/3]).
-export([stream_info/2]).
-export([tunneled_name/2]).
-export([down/1]).
-export([ws_upgrade/11]).
-export([ws_send/6]).
-record(tunnel_state, {
%% Fake socket and transport.
%% We accept 'undefined' only to simplify the init code.
socket = undefined :: #{
gun_pid := pid(),
reply_to := pid(),
stream_ref := gun:stream_ref(),
handle_continue_stream_ref := gun:stream_ref()
} | pid() | undefined,
transport = undefined :: gun_tcp_proxy | gun_tls_proxy | undefined,
%% The stream_ref from which the stream was created. When
%% the tunnel exists as a result of HTTP/2 CONNECT -> HTTP/1.1 CONNECT
%% the stream_ref is the same as the HTTP/1.1 CONNECT one.
stream_ref = undefined :: gun:stream_ref(),
%% The pid we send messages to.
reply_to = undefined :: pid(),
%% When the tunnel is a 'connect' tunnel we must dereference the
%% stream_ref. When it is 'socks' we must not as there was no
%% stream involved in creating the tunnel.
type = undefined :: connect | socks5,
%% Transport and protocol name of the tunnel layer.
tunnel_transport = undefined :: tcp | tls,
tunnel_protocol = undefined :: http | http2 | socks,
%% Tunnel information.
info = undefined :: gun:tunnel_info(),
%% The origin socket of the TLS proxy, if any. This is used to forward
%% messages to the proxy process in order to decrypt the data.
tls_origin_socket = undefined :: undefined | #{
gun_pid := pid(),
reply_to := pid(),
stream_ref := gun:stream_ref(),
handle_continue_stream_ref => gun:stream_ref()
},
opts = undefined :: undefined | any(), %% @todo Opts type.
%% Protocol module and state of the outer layer. Only initialized
%% after the TLS handshake has completed when TLS is involved.
protocol = undefined :: module(),
protocol_state = undefined :: any(),
%% When the protocol is being switched the origin may change.
%% We keep the new information to provide it in TunnelInfo of
%% the new protocol when the switch occurs.
protocol_origin = undefined :: undefined
| {origin, binary(), inet:hostname() | inet:ip_address(), inet:port_number(), connect | socks5}
}).
%% Socket is the "origin socket" and Transport the "origin transport".
%% When the Transport indicate a TLS handshake was requested, the socket
%% and transport are given to the intermediary TLS proxy process.
%%
%% Opts is the options for the underlying HTTP/2 connection,
%% with some extra information added for the tunnel.
%%
%% @todo Mark the tunnel options as reserved.
init(ReplyTo, OriginSocket, OriginTransport, Opts=#{stream_ref := StreamRef, tunnel := Tunnel},
EvHandler, EvHandlerState0) ->
#{
type := TunnelType,
transport_name := TunnelTransport,
protocol_name := TunnelProtocol,
info := TunnelInfo
} = Tunnel,
State = #tunnel_state{stream_ref=StreamRef, reply_to=ReplyTo, type=TunnelType,
tunnel_transport=TunnelTransport, tunnel_protocol=TunnelProtocol,
info=TunnelInfo, opts=maps:without([stream_ref, tunnel], Opts)},
case Tunnel of
%% Initialize the protocol.
#{new_protocol := NewProtocol} ->
{Proto, ProtoOpts} = gun_protocols:handler_and_opts(NewProtocol, Opts),
case Proto:init(ReplyTo, OriginSocket, OriginTransport,
ProtoOpts#{stream_ref => StreamRef, tunnel_transport => tcp}) of
{ok, _, ProtoState} ->
EvHandlerState = EvHandler:protocol_changed(#{
stream_ref => StreamRef,
protocol => Proto:name()
}, EvHandlerState0),
%% When the tunnel protocol is HTTP/1.1 or SOCKS
%% the gun_tunnel_up message was already sent.
_ = case TunnelProtocol of
http -> ok;
socks -> ok;
_ -> ReplyTo ! {gun_tunnel_up, self(), StreamRef, Proto:name()}
end,
{tunnel, State#tunnel_state{socket=OriginSocket, transport=OriginTransport,
protocol=Proto, protocol_state=ProtoState},
EvHandlerState};
Error={error, _} ->
Error
end;
%% We can't initialize the protocol until the TLS handshake has completed.
#{handshake_event := HandshakeEvent0, protocols := Protocols} ->
#{handle_continue_stream_ref := ContinueStreamRef} = OriginSocket,
#{
origin_host := DestHost,
origin_port := DestPort
} = TunnelInfo,
#{
tls_opts := TLSOpts,
timeout := TLSTimeout
} = HandshakeEvent0,
HandshakeEvent = HandshakeEvent0#{socket => OriginSocket},
EvHandlerState = EvHandler:tls_handshake_start(HandshakeEvent, EvHandlerState0),
{ok, ProxyPid} = gun_tls_proxy:start_link(DestHost, DestPort,
TLSOpts, TLSTimeout, OriginSocket, gun_tls_proxy_http2_connect,
{handle_continue, ContinueStreamRef, HandshakeEvent, Protocols}),
{tunnel, State#tunnel_state{socket=ProxyPid, transport=gun_tls_proxy,
tls_origin_socket=OriginSocket}, EvHandlerState}
end.
%% When we receive data we pass it forward directly for TCP;
%% or we decrypt it and pass it via handle_continue for TLS.
handle(Data, State=#tunnel_state{transport=gun_tcp_proxy,
protocol=Proto, protocol_state=ProtoState0},
CookieStore0, EvHandler, EvHandlerState0) ->
{Commands, CookieStore, EvHandlerState1} = Proto:handle(
Data, ProtoState0, CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState};
handle(Data, State=#tunnel_state{transport=gun_tls_proxy,
socket=ProxyPid, tls_origin_socket=OriginSocket},
CookieStore, _EvHandler, EvHandlerState) ->
%% When we receive a DATA frame that contains TLS-encoded data,
%% we must first forward it to the ProxyPid to be decoded. The
%% Gun process will receive it back as a tls_proxy_http2_connect
%% message and forward it to the right stream via the handle_continue
%% callback.
ProxyPid ! {tls_proxy_http2_connect, OriginSocket, Data},
{{state, State}, CookieStore, EvHandlerState}.
%% This callback will only be called for TLS.
%%
%% The StreamRef in this callback is special because it includes
%% a reference() for Socks layers as well.
handle_continue(ContinueStreamRef, {gun_tls_proxy, ProxyPid, {ok, Negotiated},
{handle_continue, _, HandshakeEvent, Protocols}},
State=#tunnel_state{socket=ProxyPid, stream_ref=StreamRef, opts=Opts},
CookieStore, EvHandler, EvHandlerState0)
when is_reference(ContinueStreamRef) ->
#{reply_to := ReplyTo} = HandshakeEvent,
NewProtocol = gun_protocols:negotiated(Negotiated, Protocols),
{Proto, ProtoOpts} = gun_protocols:handler_and_opts(NewProtocol, Opts),
EvHandlerState1 = EvHandler:tls_handshake_end(HandshakeEvent#{
socket => ProxyPid,
protocol => Proto:name()
}, EvHandlerState0),
EvHandlerState = EvHandler:protocol_changed(#{
stream_ref => StreamRef,
protocol => Proto:name()
}, EvHandlerState1),
%% @todo Terminate the current protocol or something?
OriginSocket = #{
gun_pid => self(),
reply_to => ReplyTo,
stream_ref => StreamRef
},
case Proto:init(ReplyTo, OriginSocket, gun_tcp_proxy,
ProtoOpts#{stream_ref => StreamRef, tunnel_transport => tls}) of
{ok, _, ProtoState} ->
ReplyTo ! {gun_tunnel_up, self(), StreamRef, Proto:name()},
{{state, State#tunnel_state{protocol=Proto, protocol_state=ProtoState}},
CookieStore, EvHandlerState};
Error={error, _} ->
{Error, CookieStore, EvHandlerState}
end;
handle_continue(ContinueStreamRef, {gun_tls_proxy, ProxyPid, {error, Reason},
{handle_continue, _, HandshakeEvent, _}},
#tunnel_state{socket=ProxyPid}, CookieStore, EvHandler, EvHandlerState0)
when is_reference(ContinueStreamRef) ->
EvHandlerState = EvHandler:tls_handshake_end(HandshakeEvent#{
error => Reason
}, EvHandlerState0),
%% @todo
%% The TCP connection can be closed by either peer. The END_STREAM flag
%% on a DATA frame is treated as being equivalent to the TCP FIN bit. A
%% client is expected to send a DATA frame with the END_STREAM flag set
%% after receiving a frame bearing the END_STREAM flag. A proxy that
%% receives a DATA frame with the END_STREAM flag set sends the attached
%% data with the FIN bit set on the last TCP segment. A proxy that
%% receives a TCP segment with the FIN bit set sends a DATA frame with
%% the END_STREAM flag set. Note that the final TCP segment or DATA
%% frame could be empty.
{{error, Reason}, CookieStore, EvHandlerState};
%% Send the data. This causes TLS to encrypt the data and send it to the inner layer.
handle_continue(ContinueStreamRef, {data, _ReplyTo, _StreamRef, IsFin, Data},
#tunnel_state{}, CookieStore, _EvHandler, EvHandlerState)
when is_reference(ContinueStreamRef) ->
{{send, IsFin, Data}, CookieStore, EvHandlerState};
handle_continue(ContinueStreamRef, {tls_proxy, ProxyPid, Data},
State=#tunnel_state{socket=ProxyPid, protocol=Proto, protocol_state=ProtoState},
CookieStore0, EvHandler, EvHandlerState0)
when is_reference(ContinueStreamRef) ->
{Commands, CookieStore, EvHandlerState1} = Proto:handle(
Data, ProtoState, CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState};
handle_continue(ContinueStreamRef, {tls_proxy_closed, ProxyPid},
#tunnel_state{socket=ProxyPid}, CookieStore, _EvHandler, EvHandlerState0)
when is_reference(ContinueStreamRef) ->
%% @todo All sub-streams must produce a stream_error.
{{error, closed}, CookieStore, EvHandlerState0};
handle_continue(ContinueStreamRef, {tls_proxy_error, ProxyPid, Reason},
#tunnel_state{socket=ProxyPid}, CookieStore, _EvHandler, EvHandlerState0)
when is_reference(ContinueStreamRef) ->
%% @todo All sub-streams must produce a stream_error.
{{error, Reason}, CookieStore, EvHandlerState0};
%% We always dereference the ContinueStreamRef because it includes a
%% reference() for Socks layers too.
%%
%% @todo Assert StreamRef to be our reference().
handle_continue([_StreamRef|ContinueStreamRef0], Msg,
State=#tunnel_state{protocol=Proto, protocol_state=ProtoState},
CookieStore0, EvHandler, EvHandlerState0) ->
ContinueStreamRef = case ContinueStreamRef0 of
[CSR] -> CSR;
_ -> ContinueStreamRef0
end,
{Commands, CookieStore, EvHandlerState1} = Proto:handle_continue(
ContinueStreamRef, Msg, ProtoState, CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState}.
%% @todo This function will need EvHandler/EvHandlerState?
update_flow(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState},
ReplyTo, StreamRef0, Inc) ->
StreamRef = maybe_dereference(State, StreamRef0),
Commands = Proto:update_flow(ProtoState, ReplyTo, StreamRef, Inc),
{ResCommands, undefined} = commands(Commands, State, undefined, undefined),
ResCommands.
closing(_Reason, _State, _EvHandler, EvHandlerState) ->
%% @todo Graceful shutdown must be propagated to tunnels.
{[], EvHandlerState}.
close(_Reason, _State, _EvHandler, EvHandlerState) ->
%% @todo Closing must be propagated to tunnels.
EvHandlerState.
keepalive(_State, _EvHandler, EvHandlerState) ->
%% @todo Need to figure out how to handle keepalive for tunnels.
{[], EvHandlerState}.
%% We pass the headers forward and optionally dereference StreamRef.
headers(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState0},
StreamRef0, ReplyTo, Method, Host, Port, Path, Headers,
InitialFlow, CookieStore0, EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, CookieStore, EvHandlerState1} = Proto:headers(ProtoState0, StreamRef,
ReplyTo, Method, Host, Port, Path, Headers,
InitialFlow, CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState}.
%% We pass the request forward and optionally dereference StreamRef.
request(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState0,
info=#{origin_host := OriginHost, origin_port := OriginPort}},
StreamRef0, ReplyTo, Method, _Host, _Port, Path, Headers, Body,
InitialFlow, CookieStore0, EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, CookieStore, EvHandlerState1} = Proto:request(ProtoState0, StreamRef,
ReplyTo, Method, OriginHost, OriginPort, Path, Headers, Body,
InitialFlow, CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState}.
%% When the next tunnel is SOCKS we pass the data forward directly.
%% This is needed because SOCKS has no StreamRef and the data cannot
%% therefore be passed forward through the usual method.
data(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState0,
protocol_origin={origin, _, _, _, socks5}},
StreamRef, ReplyTo, IsFin, Data, EvHandler, EvHandlerState0) ->
{Commands, EvHandlerState1} = Proto:data(ProtoState0, StreamRef,
ReplyTo, IsFin, Data, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, EvHandlerState};
%% CONNECT tunnels pass the data forward and dereference StreamRef
%% unless they are the recipient of the callback, in which case the
%% data is sent to the socket.
data(State=#tunnel_state{socket=Socket, transport=Transport,
stream_ref=TunnelStreamRef0, protocol=Proto, protocol_state=ProtoState0},
StreamRef0, ReplyTo, IsFin, Data, EvHandler, EvHandlerState0) ->
TunnelStreamRef = outer_stream_ref(TunnelStreamRef0),
case StreamRef0 of
TunnelStreamRef ->
case Transport:send(Socket, Data) of
ok -> {[], EvHandlerState0};
Error={error, _} -> {Error, EvHandlerState0}
end;
_ ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, EvHandlerState1} = Proto:data(ProtoState0, StreamRef,
ReplyTo, IsFin, Data, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State,
EvHandler, EvHandlerState1),
{ResCommands, EvHandlerState}
end.
%% We pass the CONNECT request forward and optionally dereference StreamRef.
connect(State=#tunnel_state{info=#{origin_host := Host, origin_port := Port},
protocol=Proto, protocol_state=ProtoState0},
StreamRef0, ReplyTo, Destination, _, Headers, InitialFlow,
EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, EvHandlerState1} = Proto:connect(ProtoState0, StreamRef,
ReplyTo, Destination, #{host => Host, port => Port}, Headers, InitialFlow,
EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, EvHandlerState}.
cancel(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState0},
StreamRef0, ReplyTo, EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, EvHandlerState1} = Proto:cancel(ProtoState0, StreamRef,
ReplyTo, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, EvHandlerState}.
timeout(State=#tunnel_state{protocol=Proto, protocol_state=ProtoState0}, Msg, TRef) ->
case Proto:timeout(ProtoState0, Msg, TRef) of
{state, ProtoState} ->
{state, State#tunnel_state{protocol_state=ProtoState}};
Other ->
Other
end.
stream_info(#tunnel_state{transport=Transport0, stream_ref=TunnelStreamRef, reply_to=ReplyTo,
tunnel_protocol=TunnelProtocol,
info=#{origin_host := OriginHost, origin_port := OriginPort},
protocol=Proto, protocol_state=ProtoState}, StreamRef)
when is_reference(StreamRef), TunnelProtocol =/= socks ->
Transport = case Transport0 of
gun_tcp_proxy -> tcp;
gun_tls_proxy -> tls
end,
Protocol = case Proto of
gun_tunnel -> Proto:tunneled_name(ProtoState, false);
_ -> Proto:name()
end,
{ok, #{
ref => TunnelStreamRef,
reply_to => ReplyTo,
state => running,
tunnel => #{
transport => Transport,
protocol => Protocol,
origin_scheme => case {Transport, Protocol} of
{_, raw} -> undefined;
{tcp, _} -> <<"http">>;
{tls, _} -> <<"https">>
end,
origin_host => OriginHost,
origin_port => OriginPort
}
}};
stream_info(State=#tunnel_state{type=Type,
tunnel_transport=IntermediaryTransport, tunnel_protocol=IntermediaryProtocol,
info=TunnelInfo, protocol=Proto, protocol_state=ProtoState}, StreamRef0) ->
StreamRef = maybe_dereference(State, StreamRef0),
case Proto:stream_info(ProtoState, StreamRef) of
{ok, undefined} ->
{ok, undefined};
{ok, Info} ->
#{
host := IntermediateHost,
port := IntermediatePort
} = TunnelInfo,
IntermediaryInfo = #{
type => Type,
host => IntermediateHost,
port => IntermediatePort,
transport => IntermediaryTransport,
protocol => IntermediaryProtocol
},
Intermediaries = maps:get(intermediaries, Info, []),
{ok, Info#{
intermediaries => [IntermediaryInfo|Intermediaries]
}}
end.
tunneled_name(#tunnel_state{protocol=Proto=gun_tunnel, protocol_state=ProtoState}, true) ->
Proto:tunneled_name(ProtoState, false);
tunneled_name(#tunnel_state{tunnel_protocol=TunnelProto}, false) ->
TunnelProto;
tunneled_name(#tunnel_state{protocol=Proto}, _) ->
Proto:name().
down(_State) ->
%% @todo Tunnels must be included in the gun_down message.
[].
ws_upgrade(State=#tunnel_state{info=TunnelInfo, protocol=Proto, protocol_state=ProtoState0},
StreamRef0, ReplyTo, _, _, Path, Headers, WsOpts,
CookieStore0, EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
#{
origin_host := Host,
origin_port := Port
} = TunnelInfo,
{Commands, CookieStore, EvHandlerState1} = Proto:ws_upgrade(ProtoState0, StreamRef, ReplyTo,
Host, Port, Path, Headers, WsOpts,
CookieStore0, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, CookieStore, EvHandlerState}.
ws_send(Frames, State=#tunnel_state{protocol=Proto, protocol_state=ProtoState},
StreamRef0, ReplyTo, EvHandler, EvHandlerState0) ->
StreamRef = maybe_dereference(State, StreamRef0),
{Commands, EvHandlerState1} = Proto:ws_send(Frames,
ProtoState, StreamRef, ReplyTo, EvHandler, EvHandlerState0),
{ResCommands, EvHandlerState} = commands(Commands, State, EvHandler, EvHandlerState1),
{ResCommands, EvHandlerState}.
%% Internal.
%% Returns an error on send errors, a state otherwise
commands(Command, State, EvHandler, EvHandlerState) when not is_list(Command) ->
commands([Command], State, EvHandler, EvHandlerState);
commands([], State, _, EvHandlerState) ->
{{state, State}, EvHandlerState};
commands([Error = {error, _}|_],
State=#tunnel_state{socket=Socket, transport=Transport},
_, EvHandlerState) ->
%% We must terminate the TLS proxy pid if any.
case Transport of
gun_tls_proxy -> gun_tls_proxy:close(Socket);
_ -> ok
end,
{[{state, State}, Error], EvHandlerState};
commands([{state, ProtoState}|Tail], State, EvHandler, EvHandlerState) ->
commands(Tail, State#tunnel_state{protocol_state=ProtoState}, EvHandler, EvHandlerState);
%% @todo What to do about IsFin?
commands([{send, _IsFin, Data}|Tail],
State=#tunnel_state{socket=Socket, transport=Transport},
EvHandler, EvHandlerState) ->
case Transport:send(Socket, Data) of
ok -> commands(Tail, State, EvHandler, EvHandlerState);
Error={error, _} -> {Error, EvHandlerState}
end;
commands([Origin={origin, Scheme, Host, Port, Type}|Tail],
State=#tunnel_state{stream_ref=StreamRef},
EvHandler, EvHandlerState0) ->
EvHandlerState = EvHandler:origin_changed(#{
stream_ref => StreamRef,
type => Type,
origin_scheme => Scheme,
origin_host => Host,
origin_port => Port
}, EvHandlerState0),
commands(Tail, State#tunnel_state{protocol_origin=Origin}, EvHandler, EvHandlerState);
commands([{switch_protocol, NewProtocol, ReplyTo}|Tail],
State=#tunnel_state{socket=Socket, transport=Transport, opts=Opts,
protocol_origin=undefined},
EvHandler, EvHandlerState0) ->
{Proto, ProtoOpts} = gun_protocols:handler_and_opts(NewProtocol, Opts),
%% This should only apply to Websocket for the time being.
case Proto:init(ReplyTo, Socket, Transport, ProtoOpts) of
{ok, connected_ws_only, ProtoState} ->
#{stream_ref := StreamRef} = ProtoOpts,
EvHandlerState = EvHandler:protocol_changed(#{
stream_ref => StreamRef,
protocol => Proto:name()
}, EvHandlerState0),
commands(Tail, State#tunnel_state{protocol=Proto, protocol_state=ProtoState},
EvHandler, EvHandlerState);
Error={error, _} ->
{Error, EvHandlerState0}
end;
commands([{switch_protocol, NewProtocol, ReplyTo}|Tail],
State=#tunnel_state{transport=Transport, stream_ref=TunnelStreamRef,
info=#{origin_host := Host, origin_port := Port}, opts=Opts, protocol=CurrentProto,
protocol_origin={origin, _Scheme, OriginHost, OriginPort, Type}},
EvHandler, EvHandlerState0) ->
StreamRef = case Type of
socks5 -> TunnelStreamRef;
connect -> gun_protocols:stream_ref(NewProtocol)
end,
ContinueStreamRef0 = continue_stream_ref(State),
ContinueStreamRef = case Type of
socks5 -> ContinueStreamRef0 ++ [make_ref()];
connect -> ContinueStreamRef0 ++ [lists:last(StreamRef)]
end,
OriginSocket = #{
gun_pid => self(),
reply_to => ReplyTo,
stream_ref => StreamRef,
handle_continue_stream_ref => ContinueStreamRef
},
ProtoOpts = Opts#{
stream_ref => StreamRef,
tunnel => #{
type => Type,
transport_name => case Transport of
gun_tcp_proxy -> tcp;
gun_tls_proxy -> tls
end,
protocol_name => CurrentProto:name(),
info => #{
host => Host,
port => Port,
origin_host => OriginHost,
origin_port => OriginPort
},
new_protocol => NewProtocol
}
},
Proto = gun_tunnel,
case Proto:init(ReplyTo, OriginSocket, gun_tcp_proxy, ProtoOpts, EvHandler, EvHandlerState0) of
{tunnel, ProtoState, EvHandlerState} ->
commands(Tail, State#tunnel_state{protocol=Proto, protocol_state=ProtoState},
EvHandler, EvHandlerState);
Error={error, _} ->
{Error, EvHandlerState0}
end;
commands([{tls_handshake, HandshakeEvent0, Protocols, ReplyTo}|Tail],
State=#tunnel_state{transport=Transport,
info=#{origin_host := Host, origin_port := Port}, opts=Opts, protocol=CurrentProto,
protocol_origin={origin, _Scheme, OriginHost, OriginPort, Type}},
EvHandler, EvHandlerState0) ->
#{
stream_ref := StreamRef,
tls_opts := TLSOpts0
} = HandshakeEvent0,
TLSOpts = gun:ensure_tls_opts(Protocols, TLSOpts0, OriginHost),
HandshakeEvent = HandshakeEvent0#{
tls_opts => TLSOpts
},
ContinueStreamRef0 = continue_stream_ref(State),
ContinueStreamRef = case Type of
socks5 -> ContinueStreamRef0 ++ [make_ref()];
connect -> ContinueStreamRef0 ++ [lists:last(StreamRef)]
end,
OriginSocket = #{
gun_pid => self(),
reply_to => ReplyTo,
stream_ref => StreamRef,
handle_continue_stream_ref => ContinueStreamRef
},
ProtoOpts = Opts#{
stream_ref => StreamRef,
tunnel => #{
type => Type,
transport_name => case Transport of
gun_tcp_proxy -> tcp;
gun_tls_proxy -> tls
end,
protocol_name => CurrentProto:name(),
info => #{
host => Host,
port => Port,
origin_host => OriginHost,
origin_port => OriginPort
},
handshake_event => HandshakeEvent,
protocols => Protocols
}
},
Proto = gun_tunnel,
case Proto:init(ReplyTo, OriginSocket, gun_tcp_proxy, ProtoOpts, EvHandler, EvHandlerState0) of
{tunnel, ProtoState, EvHandlerState} ->
commands(Tail, State#tunnel_state{protocol=Proto, protocol_state=ProtoState},
EvHandler, EvHandlerState);
Error={error, _} ->
{Error, EvHandlerState0}
end;
commands([{active, true}|Tail], State, EvHandler, EvHandlerState) ->
commands(Tail, State, EvHandler, EvHandlerState).
continue_stream_ref(#tunnel_state{socket=#{handle_continue_stream_ref := ContinueStreamRef}}) ->
if
is_list(ContinueStreamRef) -> ContinueStreamRef;
true -> [ContinueStreamRef]
end;
continue_stream_ref(#tunnel_state{tls_origin_socket=#{handle_continue_stream_ref := ContinueStreamRef}}) ->
if
is_list(ContinueStreamRef) -> ContinueStreamRef;
true -> [ContinueStreamRef]
end.
maybe_dereference(#tunnel_state{stream_ref=RealStreamRef, type=connect}, [StreamRef|Tail]) ->
%% We ensure that the stream_ref is correct.
StreamRef = outer_stream_ref(RealStreamRef),
case Tail of
[Ref] -> Ref;
_ -> Tail
end;
maybe_dereference(#tunnel_state{type=socks5}, StreamRef) ->
StreamRef.
outer_stream_ref(StreamRef) when is_list(StreamRef) ->
lists:last(StreamRef);
outer_stream_ref(StreamRef) ->
StreamRef.

354
gun/src/gun_ws.erl Normal file
View File

@ -0,0 +1,354 @@
%% Copyright (c) 2015-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_ws).
-export([check_options/1]).
-export([select_extensions/3]).
-export([select_protocol/2]).
-export([name/0]).
-export([opts_name/0]).
-export([has_keepalive/0]).
-export([default_keepalive/0]).
-export([init/4]).
-export([handle/5]).
-export([handle_continue/6]).
-export([update_flow/4]).
-export([closing/4]).
-export([close/4]).
-export([keepalive/3]).
-export([ws_send/6]).
-export([down/1]).
-record(payload, {
type = undefined :: cow_ws:frame_type(),
rsv = undefined :: cow_ws:rsv(),
len = undefined :: non_neg_integer(),
mask_key = undefined :: cow_ws:mask_key(),
close_code = undefined :: undefined | cow_ws:close_code(),
unmasked = <<>> :: binary(),
unmasked_len = 0 :: non_neg_integer()
}).
-record(ws_state, {
reply_to :: pid(),
stream_ref :: reference(),
socket :: inet:socket() | ssl:sslsocket(),
transport :: module(),
opts = #{} :: gun:ws_opts(),
buffer = <<>> :: binary(),
in = head :: head | #payload{} | close,
out = head :: head | close,
frag_state = undefined :: cow_ws:frag_state(),
utf8_state = 0 :: cow_ws:utf8_state(),
extensions = #{} :: cow_ws:extensions(),
flow :: integer() | infinity,
handler :: module(),
handler_state :: any()
}).
check_options(Opts) ->
do_check_options(maps:to_list(Opts)).
do_check_options([]) ->
ok;
do_check_options([{closing_timeout, infinity}|Opts]) ->
do_check_options(Opts);
do_check_options([{closing_timeout, T}|Opts]) when is_integer(T), T > 0 ->
do_check_options(Opts);
do_check_options([{compress, B}|Opts]) when is_boolean(B) ->
do_check_options(Opts);
do_check_options([{default_protocol, M}|Opts]) when is_atom(M) ->
do_check_options(Opts);
do_check_options([{flow, InitialFlow}|Opts]) when is_integer(InitialFlow), InitialFlow > 0 ->
do_check_options(Opts);
do_check_options([{keepalive, infinity}|Opts]) ->
do_check_options(Opts);
do_check_options([{keepalive, K}|Opts]) when is_integer(K), K > 0 ->
do_check_options(Opts);
do_check_options([Opt={protocols, L}|Opts]) when is_list(L) ->
case lists:usort(lists:flatten([[is_binary(B), is_atom(M)] || {B, M} <- L])) of
[true] -> do_check_options(Opts);
_ -> {error, {options, {ws, Opt}}}
end;
do_check_options([{reply_to, P}|Opts]) when is_pid(P) ->
do_check_options(Opts);
do_check_options([{silence_pings, B}|Opts]) when is_boolean(B) ->
do_check_options(Opts);
do_check_options([{user_opts, _}|Opts]) ->
do_check_options(Opts);
do_check_options([Opt|_]) ->
{error, {options, {ws, Opt}}}.
select_extensions(Headers, Extensions0, Opts) ->
case lists:keyfind(<<"sec-websocket-extensions">>, 1, Headers) of
false ->
#{};
{_, ExtHd} ->
ParsedExtHd = cow_http_hd:parse_sec_websocket_extensions(ExtHd),
validate_extensions(ParsedExtHd, Extensions0, Opts, #{})
end.
validate_extensions([], _, _, Acc) ->
Acc;
validate_extensions([{Name = <<"permessage-deflate">>, Params}|Tail], Extensions, Opts, Acc0) ->
case lists:member(Name, Extensions) of
true ->
case cow_ws:validate_permessage_deflate(Params, Acc0, Opts) of
{ok, Acc} -> validate_extensions(Tail, Extensions, Opts, Acc);
error -> close
end;
%% Fail the connection if extension was not requested.
false ->
close
end;
%% Fail the connection on unknown extension.
validate_extensions(_, _, _, _) ->
close.
%% @todo Validate protocols.
select_protocol(Headers, Opts) ->
case lists:keyfind(<<"sec-websocket-protocol">>, 1, Headers) of
false ->
maps:get(default_protocol, Opts, gun_ws_h);
{_, Proto} ->
ProtoOpt = maps:get(protocols, Opts, []),
case lists:keyfind(Proto, 1, ProtoOpt) of
{_, Handler} ->
Handler;
false ->
close
end
end.
name() -> ws.
opts_name() -> ws_opts.
has_keepalive() -> true.
default_keepalive() -> infinity.
init(ReplyTo, Socket, Transport, #{stream_ref := StreamRef, headers := Headers,
extensions := Extensions, flow := InitialFlow, handler := Handler, opts := Opts}) ->
{ok, HandlerState} = Handler:init(ReplyTo, StreamRef, Headers, Opts),
{ok, connected_ws_only, #ws_state{reply_to=ReplyTo, stream_ref=StreamRef,
socket=Socket, transport=Transport, opts=Opts, extensions=Extensions,
flow=InitialFlow, handler=Handler, handler_state=HandlerState}}.
handle(Data, State, CookieStore, EvHandler, EvHandlerState0) ->
{Commands, EvHandlerState} = handle(Data, State, EvHandler, EvHandlerState0),
{Commands, CookieStore, EvHandlerState}.
%% Do not handle anything if we received a close frame.
%% Initiate or terminate the closing state depending on whether we sent a close yet.
handle(_, State=#ws_state{in=close, out=close}, _, EvHandlerState) ->
{[{state, State}, close], EvHandlerState};
handle(_, State=#ws_state{in=close}, EvHandler, EvHandlerState) ->
closing(normal, State, EvHandler, EvHandlerState);
%% Shortcut for common case when Data is empty after processing a frame.
handle(<<>>, State=#ws_state{in=head}, _, EvHandlerState) ->
maybe_active(State, EvHandlerState);
handle(Data, State=#ws_state{reply_to=ReplyTo, stream_ref=StreamRef, buffer=Buffer,
in=head, frag_state=FragState, extensions=Extensions},
EvHandler, EvHandlerState0) ->
%% Send the event only if there was no data in the buffer.
%% If there is data in the buffer then we already sent the event.
EvHandlerState1 = case Buffer of
<<>> ->
EvHandler:ws_recv_frame_start(#{
stream_ref => StreamRef,
reply_to => ReplyTo,
frag_state => FragState,
extensions => Extensions
}, EvHandlerState0);
_ ->
EvHandlerState0
end,
Data2 = << Buffer/binary, Data/binary >>,
case cow_ws:parse_header(Data2, Extensions, FragState) of
{Type, FragState2, Rsv, Len, MaskKey, Rest} ->
EvHandlerState = EvHandler:ws_recv_frame_header(#{
stream_ref => StreamRef,
reply_to => ReplyTo,
frag_state => FragState2,
extensions => Extensions,
type => Type,
rsv => Rsv,
len => Len,
mask_key => MaskKey
}, EvHandlerState1),
handle(Rest, State#ws_state{buffer= <<>>,
in=#payload{type=Type, rsv=Rsv, len=Len, mask_key=MaskKey},
frag_state=FragState2}, EvHandler, EvHandlerState);
more ->
maybe_active(State#ws_state{buffer=Data2}, EvHandlerState1);
error ->
closing({error, badframe}, State, EvHandler, EvHandlerState1)
end;
handle(Data, State=#ws_state{in=In=#payload{type=Type, rsv=Rsv, len=Len, mask_key=MaskKey,
close_code=CloseCode, unmasked=Unmasked, unmasked_len=UnmaskedLen}, frag_state=FragState,
utf8_state=Utf8State, extensions=Extensions}, EvHandler, EvHandlerState) ->
case cow_ws:parse_payload(Data, MaskKey, Utf8State, UnmaskedLen, Type, Len, FragState, Extensions, Rsv) of
{ok, CloseCode2, Payload, Utf8State2, Rest} ->
dispatch(Rest, State#ws_state{in=head, utf8_state=Utf8State2}, Type,
<<Unmasked/binary, Payload/binary>>, CloseCode2,
EvHandler, EvHandlerState);
{ok, Payload, Utf8State2, Rest} ->
dispatch(Rest, State#ws_state{in=head, utf8_state=Utf8State2}, Type,
<<Unmasked/binary, Payload/binary>>, CloseCode,
EvHandler, EvHandlerState);
{more, CloseCode2, Payload, Utf8State2} ->
maybe_active(State#ws_state{in=In#payload{close_code=CloseCode2,
unmasked= <<Unmasked/binary, Payload/binary>>,
len=Len - byte_size(Data), unmasked_len=2 + byte_size(Data)}, utf8_state=Utf8State2},
EvHandlerState);
{more, Payload, Utf8State2} ->
maybe_active(State#ws_state{in=In#payload{unmasked= <<Unmasked/binary, Payload/binary>>,
len=Len - byte_size(Data), unmasked_len=UnmaskedLen + byte_size(Data)}, utf8_state=Utf8State2},
EvHandlerState);
Error = {error, _Reason} ->
closing(Error, State, EvHandler, EvHandlerState)
end.
handle_continue(ContinueStreamRef, {data, _ReplyTo, _StreamRef, IsFin, Data},
#ws_state{}, CookieStore, _EvHandler, EvHandlerState)
when is_reference(ContinueStreamRef) ->
{{send, IsFin, Data}, CookieStore, EvHandlerState}.
maybe_active(State=#ws_state{flow=Flow}, EvHandlerState) ->
{[
{state, State},
{active, Flow > 0}
], EvHandlerState}.
dispatch(Rest, State0=#ws_state{reply_to=ReplyTo, stream_ref=StreamRef,
frag_state=FragState, extensions=Extensions, flow=Flow0,
handler=Handler, handler_state=HandlerState0},
Type, Payload, CloseCode, EvHandler, EvHandlerState0) ->
EvHandlerState1 = EvHandler:ws_recv_frame_end(#{
stream_ref => StreamRef,
reply_to => ReplyTo,
extensions => Extensions,
close_code => CloseCode,
payload => Payload
}, EvHandlerState0),
case cow_ws:make_frame(Type, Payload, CloseCode, FragState) of
Frame ->
{ok, Dec, HandlerState} = Handler:handle(Frame, HandlerState0),
Flow = case Flow0 of
infinity -> infinity;
_ -> Flow0 - Dec
end,
State1 = State0#ws_state{flow=Flow, handler_state=HandlerState},
{State, EvHandlerState} = case Frame of
ping ->
{[], EvHandlerState2} = send(pong, State1, ReplyTo, EvHandler, EvHandlerState1),
{State1, EvHandlerState2};
{ping, Payload} ->
{[], EvHandlerState2} = send({pong, Payload}, State1, ReplyTo, EvHandler, EvHandlerState1),
{State1, EvHandlerState2};
close ->
{State1#ws_state{in=close}, EvHandlerState1};
{close, _, _} ->
{State1#ws_state{in=close}, EvHandlerState1};
{fragment, fin, _, _} ->
{State1#ws_state{frag_state=undefined}, EvHandlerState1};
_ ->
{State1, EvHandlerState1}
end,
handle(Rest, State, EvHandler, EvHandlerState)
end.
update_flow(State=#ws_state{flow=Flow0}, _ReplyTo, _StreamRef, Inc) ->
Flow = case Flow0 of
infinity -> infinity;
_ -> Flow0 + Inc
end,
[
{state, State#ws_state{flow=Flow}},
{active, Flow > 0}
].
%% The user already sent the close frame; do nothing.
closing(_, State=#ws_state{out=close}, _, EvHandlerState) ->
{closing(State), EvHandlerState};
closing(Reason, State=#ws_state{reply_to=ReplyTo}, EvHandler, EvHandlerState) ->
Code = case Reason of
normal -> 1000;
owner_down -> 1001;
shutdown -> 1001;
{error, badframe} -> 1002;
{error, badencoding} -> 1007
end,
send({close, Code, <<>>}, State, ReplyTo, EvHandler, EvHandlerState).
closing(#ws_state{opts=Opts}) ->
Timeout = maps:get(closing_timeout, Opts, 15000),
{closing, Timeout}.
close(_, _, _, EvHandlerState) ->
EvHandlerState.
keepalive(State=#ws_state{reply_to=ReplyTo}, EvHandler, EvHandlerState0) ->
send(ping, State, ReplyTo, EvHandler, EvHandlerState0).
%% Send one frame.
send(Frame, State=#ws_state{stream_ref=StreamRef,
socket=Socket, transport=Transport, in=In, extensions=Extensions},
ReplyTo, EvHandler, EvHandlerState0) ->
WsSendFrameEvent = #{
stream_ref => StreamRef,
reply_to => ReplyTo,
extensions => Extensions,
frame => Frame
},
EvHandlerState1 = EvHandler:ws_send_frame_start(WsSendFrameEvent, EvHandlerState0),
case Transport:send(Socket, cow_ws:masked_frame(Frame, Extensions)) of
ok ->
EvHandlerState = EvHandler:ws_send_frame_end(WsSendFrameEvent, EvHandlerState1),
if
Frame =:= close; element(1, Frame) =:= close ->
{[
{state, State#ws_state{out=close}},
%% We can close immediately if we already
%% received a close frame.
case In of
close -> close;
_ -> closing(State)
end
], EvHandlerState};
true ->
{[], EvHandlerState}
end;
Error={error, _} ->
{Error, EvHandlerState0}
end.
%% Send many frames.
ws_send(Frame, State, ReplyTo, EvHandler, EvHandlerState) when not is_list(Frame) ->
send(Frame, State, ReplyTo, EvHandler, EvHandlerState);
ws_send([], _, _, _, EvHandlerState) ->
{[], EvHandlerState};
ws_send([Frame|Tail], State, ReplyTo, EvHandler, EvHandlerState0) ->
case send(Frame, State, ReplyTo, EvHandler, EvHandlerState0) of
{[], EvHandlerState} ->
ws_send(Tail, State, ReplyTo, EvHandler, EvHandlerState);
Other ->
Other
end.
%% @todo We should probably check the _StreamRef value.
ws_send(Frames, State, _StreamRef, ReplyTo, EvHandler, EvHandlerState) ->
ws_send(Frames, State, ReplyTo, EvHandler, EvHandlerState).
down(#ws_state{stream_ref=StreamRef}) ->
[StreamRef].

44
gun/src/gun_ws_h.erl Normal file
View File

@ -0,0 +1,44 @@
%% Copyright (c) 2017-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_ws_h).
-behavior(gun_ws_protocol).
-export([init/4]).
-export([handle/2]).
-record(state, {
reply_to :: pid(),
stream_ref :: reference(),
frag_buffer = <<>> :: binary(),
silence_pings :: boolean()
}).
init(ReplyTo, StreamRef, _, Opts) ->
{ok, #state{reply_to=ReplyTo, stream_ref=StreamRef,
silence_pings=maps:get(silence_pings, Opts, true)}}.
handle({fragment, nofin, _, Payload},
State=#state{frag_buffer=SoFar}) ->
{ok, 0, State#state{frag_buffer= << SoFar/binary, Payload/binary >>}};
handle({fragment, fin, Type, Payload},
State=#state{reply_to=ReplyTo, stream_ref=StreamRef, frag_buffer=SoFar}) ->
ReplyTo ! {gun_ws, self(), StreamRef, {Type, << SoFar/binary, Payload/binary >>}},
{ok, 1, State#state{frag_buffer= <<>>}};
handle(Frame, State=#state{silence_pings=true}) when Frame =:= ping; Frame =:= pong;
element(1, Frame) =:= ping; element(1, Frame) =:= pong ->
{ok, 0, State};
handle(Frame, State=#state{reply_to=ReplyTo, stream_ref=StreamRef}) ->
ReplyTo ! {gun_ws, self(), StreamRef, Frame},
{ok, 1, State}.

25
gun/src/gun_ws_protocol.erl Normal file
View File

@ -0,0 +1,25 @@
%% Copyright (c) 2022-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_ws_protocol).
-type protocol_state() :: any().
-export_type([protocol_state/0]).
-callback init(pid(), reference(), cow_http:headers(), gun:ws_opts())
-> {ok, protocol_state()}.
-callback handle(cow_ws:frame(), State)
-> {ok, non_neg_integer(), State}
when State::protocol_state().

2170
gun/test/event_SUITE.erl Normal file
View File

File diff suppressed because it is too large Load Diff

339
gun/test/flow_SUITE.erl Normal file
View File

@ -0,0 +1,339 @@
%% Copyright (c) 2019-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(flow_SUITE).
-compile(export_all).
-compile(nowarn_export_all).
-import(ct_helper, [doc/1]).
all() ->
[{group, flow}].
groups() ->
[{flow, [parallel], ct_helper:all(?MODULE)}].
%% Tests.
default_flow_http(_) ->
doc("Confirm flow control default can be changed and overriden for HTTP/1.1."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, date}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
%% First we check that we can set the flow for the entire connection.
{ok, ConnPid1} = gun:open("localhost", Port, #{
http_opts => #{flow => 1}
}),
{ok, http} = gun:await_up(ConnPid1),
StreamRef1 = gun:get(ConnPid1, "/"),
{response, nofin, 200, _} = gun:await(ConnPid1, StreamRef1),
{data, nofin, _} = gun:await(ConnPid1, StreamRef1),
{error, timeout} = gun:await(ConnPid1, StreamRef1, 1500),
gun:close(ConnPid1),
%% Then we confirm that we can override it per request.
{ok, ConnPid2} = gun:open("localhost", Port, #{
http_opts => #{flow => 1}
}),
{ok, http} = gun:await_up(ConnPid2),
StreamRef2 = gun:get(ConnPid2, "/", [], #{flow => 2}),
{response, nofin, 200, _} = gun:await(ConnPid2, StreamRef2),
{data, nofin, _} = gun:await(ConnPid2, StreamRef2),
{data, nofin, _} = gun:await(ConnPid2, StreamRef2),
{error, timeout} = gun:await(ConnPid2, StreamRef2, 1500),
gun:close(ConnPid2)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
default_flow_http2(_) ->
doc("Confirm flow control default can be changed and overriden for HTTP/2."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, 40000}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
%% First we check that we can set the flow for the entire connection.
{ok, ConnPid} = gun:open("localhost", Port, #{
http2_opts => #{
flow => 1,
%% We set the max frame size to the same as the initial
%% window size in order to reduce the number of data messages.
initial_connection_window_size => 65535,
initial_stream_window_size => 65535,
max_frame_size_received => 65535
},
protocols => [http2]
}),
{ok, http2} = gun:await_up(ConnPid),
StreamRef1 = gun:get(ConnPid, "/"),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef1),
%% We set the flow to 1 therefore we will receive *2* data messages,
%% and then nothing because the window was fully consumed.
{data, nofin, _} = gun:await(ConnPid, StreamRef1),
{data, nofin, _} = gun:await(ConnPid, StreamRef1),
{error, timeout} = gun:await(ConnPid, StreamRef1, 1500),
%% Then we confirm that we can override it per request.
StreamRef2 = gun:get(ConnPid, "/", [], #{flow => 2}),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef2),
%% We set the flow to 2 but due to the ensure_window algorithm
%% we end up receiving *4* data messages before we consume
%% the window.
{data, nofin, _} = gun:await(ConnPid, StreamRef2),
{data, nofin, _} = gun:await(ConnPid, StreamRef2),
{data, nofin, _} = gun:await(ConnPid, StreamRef2),
{data, nofin, _} = gun:await(ConnPid, StreamRef2),
{error, timeout} = gun:await(ConnPid, StreamRef2, 1500),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
flow_http(_) ->
doc("Confirm flow control works as intended for HTTP/1.1."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, date}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", [], #{flow => 1}),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
%% We set the flow to 1 therefore we will receive 1 data message,
%% and then nothing because Gun doesn't read from the socket.
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 3000),
%% We then update the flow and get 2 more data messages but no more.
gun:update_flow(ConnPid, StreamRef, 2),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 1000),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
flow_http2(_) ->
doc("Confirm flow control works as intended for HTTP/2."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, 40000}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port, #{
%% We set the max frame size to the same as the initial
%% window size in order to reduce the number of data messages.
http2_opts => #{
initial_connection_window_size => 65535,
initial_stream_window_size => 65535,
max_frame_size_received => 65535
},
protocols => [http2]
}),
{ok, http2} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", [], #{flow => 1}),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
%% We set the flow to 1 therefore we will receive *2* data messages,
%% and then nothing because the window was fully consumed.
{data, nofin, D1} = gun:await(ConnPid, StreamRef),
{data, nofin, D2} = gun:await(ConnPid, StreamRef),
%% We consumed all the window available.
65535 = byte_size(D1) + byte_size(D2),
{error, timeout} = gun:await(ConnPid, StreamRef, 3500),
%% We then update the flow and get *5* more data messages but no more.
gun:update_flow(ConnPid, StreamRef, 2),
{data, nofin, D3} = gun:await(ConnPid, StreamRef),
{data, nofin, D4} = gun:await(ConnPid, StreamRef),
{data, nofin, D5} = gun:await(ConnPid, StreamRef),
{data, nofin, D6} = gun:await(ConnPid, StreamRef),
{data, nofin, D7} = gun:await(ConnPid, StreamRef),
%% We consumed all the window available again.
%% D3 is the end of the truncated D2, D4, D5 and D6 are full and D7 truncated.
131070 = byte_size(D3) + byte_size(D4) + byte_size(D5) + byte_size(D6) + byte_size(D7),
{error, timeout} = gun:await(ConnPid, StreamRef, 1000),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
flow_ws(_) ->
doc("Confirm flow control works as intended for Websocket."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", ws_echo_h, []}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:ws_upgrade(ConnPid, "/", [], #{flow => 1}),
{upgrade, [<<"websocket">>], _} = gun:await(ConnPid, StreamRef),
%% We send 2 frames with some time in between to make sure that
%% Gun handles them in separate Protocol:handle calls.
Frame = {text, <<"Hello!">>},
gun:ws_send(ConnPid, StreamRef, Frame),
timer:sleep(500),
gun:ws_send(ConnPid, StreamRef, Frame),
%% We set the flow to 1 therefore we will receive 1 data message,
%% and then nothing because Gun doesn't read from the socket.
{ws, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 3000),
%% We then update the flow, send 2 frames with some time in between
%% and get 2 more data messages but no more.
gun:update_flow(ConnPid, StreamRef, 2),
gun:ws_send(ConnPid, StreamRef, Frame),
timer:sleep(500),
gun:ws_send(ConnPid, StreamRef, Frame),
{ws, _} = gun:await(ConnPid, StreamRef),
{ws, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 1000),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
no_flow_http(_) ->
doc("Ignore flow updates for no-flow streams for HTTP/1.1."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, date}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", []),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
gun:update_flow(ConnPid, StreamRef, 2),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
no_flow_http2(_) ->
doc("Ignore flow updates for no-flow streams for HTTP/2."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, date}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port, #{
protocols => [http2]
}),
{ok, http2} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", []),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
gun:update_flow(ConnPid, StreamRef, 2),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
{data, nofin, _} = gun:await(ConnPid, StreamRef),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
no_flow_ws(_) ->
doc("Ignore flow updates for no-flow streams for Websocket."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", ws_echo_h, []}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:ws_upgrade(ConnPid, "/", []),
{upgrade, [<<"websocket">>], _} = gun:await(ConnPid, StreamRef),
gun:update_flow(ConnPid, StreamRef, 2),
Frame = {text, <<"Hello!">>},
gun:ws_send(ConnPid, StreamRef, Frame),
timer:sleep(100),
gun:ws_send(ConnPid, StreamRef, Frame),
{ws, _} = gun:await(ConnPid, StreamRef),
{ws, _} = gun:await(ConnPid, StreamRef),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
sse_flow_http(_) ->
doc("Confirm flow control works as intended for HTTP/1.1 "
"when using the gun_sse_h content handler."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, date}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port, #{
http_opts => #{content_handlers => [gun_sse_h, gun_data_h]}
}),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", [], #{flow => 1}),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
%% We set the flow to 1 therefore we will receive 1 event message,
%% and then nothing because Gun doesn't read from the socket. We
%% set the timeout to 2500 to ensure there is only going to be one
%% message queued up.
{sse, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 2500),
%% We then update the flow and get 2 more event messages but no more.
gun:update_flow(ConnPid, StreamRef, 2),
{sse, _} = gun:await(ConnPid, StreamRef),
{sse, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 1000),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.
sse_flow_http2(_) ->
doc("Confirm flow control works as intended for HTTP/2 "
"when using the gun_sse_h content handler."),
{ok, _} = cowboy:start_clear(?FUNCTION_NAME, [], #{env => #{
dispatch => cowboy_router:compile([{'_', [{"/", sse_clock_h, 40000}]}])
}}),
Port = ranch:get_port(?FUNCTION_NAME),
try
{ok, ConnPid} = gun:open("localhost", Port, #{
%% We set the max frame size to the same as the initial
%% window size in order to reduce the number of data messages.
http2_opts => #{
content_handlers => [gun_sse_h, gun_data_h],
initial_connection_window_size => 65535,
initial_stream_window_size => 65535,
max_frame_size_received => 65535
},
protocols => [http2]
}),
{ok, http2} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/", [], #{flow => 1}),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
%% We set the flow to 1 therefore we will receive 1 event message,
%% and then nothing because the window was fully consumed before
%% the second event was fully received.
{sse, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 3000),
%% We then update the flow and get 3 more event messages but no more.
%% We get an extra message because of the ensure_window algorithm.
gun:update_flow(ConnPid, StreamRef, 2),
{sse, _} = gun:await(ConnPid, StreamRef),
{sse, _} = gun:await(ConnPid, StreamRef),
{sse, _} = gun:await(ConnPid, StreamRef),
{error, timeout} = gun:await(ConnPid, StreamRef, 1000),
gun:close(ConnPid)
after
cowboy:stop_listener(?FUNCTION_NAME)
end.

754
gun/test/gun_SUITE.erl Normal file
View File

@ -0,0 +1,754 @@
%% Copyright (c) 2017-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_SUITE).
-compile(export_all).
-compile(nowarn_export_all).
-import(ct_helper, [doc/1]).
-import(ct_helper, [name/0]).
-import(gun_test, [init_origin/2]).
-import(gun_test, [init_origin/3]).
-import(gun_test, [receive_from/1]).
-import(gun_test, [receive_all_from/2]).
-import(gun_test_event_h, [receive_event/1]).
-import(gun_test_event_h, [receive_event/2]).
suite() ->
[{timetrap, 30000}].
all() ->
[{group, gun}].
groups() ->
[{gun, [parallel], ct_helper:all(?MODULE)}].
%% Tests.
atom_header_name(_) ->
doc("Header names may be given as atom."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
_ = gun:get(Pid, "/", [
{'User-Agent', "Gun/atom-headers"}
]),
Data = receive_from(OriginPid),
Lines = binary:split(Data, <<"\r\n">>, [global]),
[<<"user-agent: Gun/atom-headers">>] = [L || <<"user-agent: ", _/bits>> = L <- Lines],
gun:close(Pid).
atom_hostname(_) ->
doc("Hostnames may be given as atom."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open('localhost', OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
_ = gun:get(Pid, "/"),
Data = receive_from(OriginPid),
Lines = binary:split(Data, <<"\r\n">>, [global]),
[<<"host: localhost:", _/bits>>] = [L || <<"host: ", _/bits>> = L <- Lines],
gun:close(Pid).
connect_timeout(_) ->
doc("Ensure an integer value for connect_timeout is accepted."),
do_timeout(connect_timeout, 1000).
connect_timeout_infinity(_) ->
doc("Ensure infinity for connect_timeout is accepted."),
do_timeout(connect_timeout, infinity).
domain_lookup_timeout(_) ->
doc("Ensure an integer value for domain_lookup_timeout is accepted."),
do_timeout(domain_lookup_timeout, 1000).
domain_lookup_timeout_infinity(_) ->
doc("Ensure infinity for domain_lookup_timeout is accepted."),
do_timeout(domain_lookup_timeout, infinity).
do_timeout(Opt, Timeout) ->
{ok, ConnPid} = gun:open("localhost", 12345, #{
Opt => Timeout,
event_handler => {gun_test_event_h, self()},
retry => 0
}),
%% The connection will not succeed. We will however receive
%% an init event from the connection process that indicates
%% that the timeout value was accepted, since the timeout
%% checks occur earlier.
{_, init, _} = receive_event(ConnPid),
gun:close(ConnPid).
ignore_empty_data_http(_) ->
doc("When gun:data/4 is called with nofin and empty data, it must be ignored."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
Ref = gun:put(Pid, "/", []),
gun:data(Pid, Ref, nofin, "hello "),
gun:data(Pid, Ref, nofin, ["", <<>>]),
gun:data(Pid, Ref, fin, "world!"),
Data = receive_all_from(OriginPid, 500),
Lines = binary:split(Data, <<"\r\n">>, [global]),
Zero = [Z || <<"0">> = Z <- Lines],
1 = length(Zero),
gun:close(Pid).
ignore_empty_data_fin_http(_) ->
doc("When gun:data/4 is called with fin and empty data, it must send a final empty chunk."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
Ref = gun:post(Pid, "/", []),
gun:data(Pid, Ref, nofin, "hello"),
gun:data(Pid, Ref, fin, ["", <<>>]),
Data = receive_all_from(OriginPid, 500),
Lines = binary:split(Data, <<"\r\n">>, [global]),
Zero = [Z || <<"0">> = Z <- Lines],
1 = length(Zero),
gun:close(Pid).
ignore_empty_data_http2(_) ->
doc("When gun:data/4 is called with nofin and empty data, it must be ignored."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http2),
{ok, Pid} = gun:open("localhost", OriginPort, #{protocols => [http2]}),
{ok, http2} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
Ref = gun:put(Pid, "/", []),
gun:data(Pid, Ref, nofin, "hello "),
gun:data(Pid, Ref, nofin, ["", <<>>]),
gun:data(Pid, Ref, fin, "world!"),
Data = receive_all_from(OriginPid, 500),
<<
%% HEADERS frame.
Len1:24, 1, _:40, _:Len1/unit:8,
%% First DATA frame.
6:24, 0, _:7, 0:1, _:32, "hello ",
%% Second and final DATA frame.
6:24, 0, _:7, 1:1, _:32, "world!"
>> = Data,
gun:close(Pid).
ignore_empty_data_fin_http2(_) ->
doc("When gun:data/4 is called with fin and empty data, it must send a final empty DATA frame."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http2),
{ok, Pid} = gun:open("localhost", OriginPort, #{protocols => [http2]}),
{ok, http2} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
Ref = gun:put(Pid, "/", []),
gun:data(Pid, Ref, nofin, "hello "),
gun:data(Pid, Ref, nofin, "world!"),
gun:data(Pid, Ref, fin, ["", <<>>]),
Data = receive_all_from(OriginPid, 500),
<<
%% HEADERS frame.
Len1:24, 1, _:40, _:Len1/unit:8,
%% First DATA frame.
6:24, 0, _:7, 0:1, _:32, "hello ",
%% Second DATA frame.
6:24, 0, _:7, 0:1, _:32, "world!",
%% Final empty DATA frame.
0:24, 0, _:7, 1:1, _:32
>> = Data,
gun:close(Pid).
info(_) ->
doc("Get info from the Gun connection."),
{ok, ListenSocket} = gen_tcp:listen(0, [binary, {active, false}]),
{ok, {_, Port}} = inet:sockname(ListenSocket),
{ok, Pid} = gun:open("localhost", Port),
{ok, _} = gen_tcp:accept(ListenSocket, 5000),
#{sock_ip := _, sock_port := _} = gun:info(Pid),
gun:close(Pid).
keepalive_infinity(_) ->
doc("Ensure infinity for keepalive is accepted by all protocols."),
{ok, ConnPid} = gun:open("localhost", 12345, #{
event_handler => {gun_test_event_h, self()},
http_opts => #{keepalive => infinity},
http2_opts => #{keepalive => infinity},
retry => 0
}),
%% The connection will not succeed. We will however receive
%% an init event from the connection process that indicates
%% that the timeout value was accepted, since the timeout
%% checks occur earlier.
{_, init, _} = receive_event(ConnPid),
gun:close(ConnPid).
killed_streams_http(_) ->
doc("Ensure completed responses with a connection: close are not considered killed streams."),
{ok, _, OriginPort} = init_origin(tcp, http,
fun (_, _, ClientSocket, ClientTransport) ->
{ok, _} = ClientTransport:recv(ClientSocket, 0, 1000),
ClientTransport:send(ClientSocket,
"HTTP/1.1 200 OK\r\n"
"connection: close\r\n"
"content-length: 12\r\n"
"\r\n"
"hello world!"
)
end),
{ok, ConnPid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(ConnPid),
StreamRef = gun:get(ConnPid, "/"),
{response, nofin, 200, _} = gun:await(ConnPid, StreamRef),
{ok, <<"hello world!">>} = gun:await_body(ConnPid, StreamRef),
receive
{gun_down, ConnPid, http, normal, KilledStreams} ->
[] = KilledStreams,
gun:close(ConnPid)
end.
list_header_name(_) ->
doc("Header names may be given as list."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
_ = gun:get(Pid, "/", [
{"User-Agent", "Gun/list-headers"}
]),
Data = receive_from(OriginPid),
Lines = binary:split(Data, <<"\r\n">>, [global]),
[<<"user-agent: Gun/list-headers">>] = [L || <<"user-agent: ", _/bits>> = L <- Lines],
gun:close(Pid).
map_headers(_) ->
doc("Header names may be given as a map."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
_ = gun:get(Pid, "/", #{
<<"USER-agent">> => "Gun/map-headers"
}),
Data = receive_from(OriginPid),
Lines = binary:split(Data, <<"\r\n">>, [global]),
[<<"user-agent: Gun/map-headers">>] = [L || <<"user-agent: ", _/bits>> = L <- Lines],
gun:close(Pid).
postpone_request_while_not_connected(_) ->
doc("Ensure Gun doesn't raise error when requesting in retries"),
%% Try connecting to a server that isn't up yet.
{ok, ConnPid} = gun:open("localhost", 23456, #{
event_handler => {gun_test_event_h, self()},
retry => 5,
retry_timeout => 1000
}),
_ = gun:get(ConnPid, "/postponed"),
%% Wait for the connection attempt. to fail.
{_, connect_end, #{error := _}} = receive_event(ConnPid, connect_end),
%% Start the server so that next retry will result in the client connecting successfully.
{ok, ListenSocket} = gen_tcp:listen(23456, [binary, {active, false}, {reuseaddr, true}]),
{ok, ClientSocket} = gen_tcp:accept(ListenSocket, 5000),
%% The client should now be up.
{ok, http} = gun:await_up(ConnPid),
%% The server receives the postponed request.
{ok, <<"GET /postponed HTTP/1.1\r\n", _/bits>>} = gen_tcp:recv(ClientSocket, 0, 5000),
gun:close(ConnPid).
reply_to_http(_) ->
doc("The reply_to option allows using a separate process for requests."),
do_reply_to(http).
reply_to_http2(_) ->
doc("The reply_to option allows using a separate process for requests."),
do_reply_to(http2).
do_reply_to(Protocol) ->
{ok, OriginPid, OriginPort} = init_origin(tcp, Protocol,
fun(_, _, ClientSocket, ClientTransport) ->
{ok, _} = ClientTransport:recv(ClientSocket, 0, infinity),
ResponseData = case Protocol of
http ->
"HTTP/1.1 200 OK\r\n"
"Content-length: 12\r\n"
"\r\n"
"Hello world!";
http2 ->
%% Send a HEADERS frame with PRIORITY back.
{HeadersBlock, _} = cow_hpack:encode([
{<<":status">>, <<"200">>}
]),
Len = iolist_size(HeadersBlock),
[
<<Len:24, 1:8,
0:2, %% Undefined.
0:1, %% PRIORITY.
0:1, %% Undefined.
0:1, %% PADDED.
1:1, %% END_HEADERS.
0:1, %% Undefined.
1:1, %% END_STREAM.
0:1, 1:31>>,
HeadersBlock
]
end,
ok = ClientTransport:send(ClientSocket, ResponseData),
timer:sleep(1000)
end),
{ok, Pid} = gun:open("localhost", OriginPort, #{protocols => [Protocol]}),
{ok, Protocol} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
Self = self(),
ReplyTo = spawn(fun() ->
receive Ref when is_reference(Ref) ->
Response = gun:await(Pid, Ref, infinity),
Self ! Response
end
end),
Ref = gun:get(Pid, "/", [], #{reply_to => ReplyTo}),
ReplyTo ! Ref,
receive
Msg ->
{response, _, _, _} = Msg,
gun:close(Pid)
end.
retry_0(_) ->
doc("Ensure Gun gives up immediately with retry=0."),
{ok, ConnPid} = gun:open("localhost", 12345, #{
event_handler => {gun_test_event_h, self()},
retry => 0,
retry_timeout => 500
}),
{_, init, _} = receive_event(ConnPid),
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
{_, terminate, _} = receive_event(ConnPid),
ok.
retry_0_disconnect(_) ->
doc("Ensure Gun gives up immediately with retry=0 after a successful connection."),
{ok, ListenSocket} = gen_tcp:listen(0, [binary, {active, false}]),
{ok, {_, Port}} = inet:sockname(ListenSocket),
{ok, ConnPid} = gun:open("localhost", Port, #{
event_handler => {gun_test_event_h, self()},
retry => 0,
retry_timeout => 500
}),
{_, init, _} = receive_event(ConnPid),
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
%% We accept the connection and then close it to trigger a disconnect.
{ok, ClientSocket} = gen_tcp:accept(ListenSocket, 5000),
gen_tcp:close(ClientSocket),
%% Connection was successful.
{_, connect_end, ConnectEndEvent} = receive_event(ConnPid),
false = maps:is_key(error, ConnectEndEvent),
%% When the connection is closed we terminate immediately.
{_, disconnect, _} = receive_event(ConnPid),
{_, terminate, _} = receive_event(ConnPid),
ok.
retry_1(_) ->
doc("Ensure Gun gives up with retry=1."),
{ok, ConnPid} = gun:open("localhost", 12345, #{
event_handler => {gun_test_event_h, self()},
retry => 1,
retry_timeout => 500
}),
{_, init, _} = receive_event(ConnPid),
%% Initial attempt.
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
%% Retry.
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
{_, terminate, _} = receive_event(ConnPid),
ok.
retry_1_disconnect(_) ->
doc("Ensure Gun doesn't give up with retry=1 after a successful connection "
"and attempts to reconnect immediately, ignoring retry_timeout."),
{ok, ListenSocket} = gen_tcp:listen(0, [binary, {active, false}]),
{ok, {_, Port}} = inet:sockname(ListenSocket),
{ok, ConnPid} = gun:open("localhost", Port, #{
event_handler => {gun_test_event_h, self()},
retry => 1,
retry_timeout => 30000
}),
{_, init, _} = receive_event(ConnPid),
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
%% We accept the connection and then close it to trigger a disconnect.
{ok, ClientSocket} = gen_tcp:accept(ListenSocket, 5000),
gen_tcp:close(ClientSocket),
%% Connection was successful.
{_, connect_end, ConnectEndEvent} = receive_event(ConnPid),
false = maps:is_key(error, ConnectEndEvent),
%% We confirm that Gun reconnects before the retry timeout,
%% as it is ignored on the first reconnection.
{ok, _} = gen_tcp:accept(ListenSocket, 5000),
gun:close(ConnPid).
retry_fun(_) ->
doc("Ensure the retry_fun is used when provided."),
{ok, ConnPid} = gun:open("localhost", 12345, #{
event_handler => {gun_test_event_h, self()},
retry => 5,
retry_fun => fun(_, _) -> #{retries => 0, timeout => 0} end,
retry_timeout => 60000
}),
{_, init, _} = receive_event(ConnPid),
%% Initial attempt.
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
%% When retry is not disabled (retry!=0) we necessarily
%% have at least one retry attempt using the fun.
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
{_, terminate, _} = receive_event(ConnPid),
ok.
retry_timeout(_) ->
doc("Ensure the retry_timeout value is enforced. The first retry is immediate "
"and therefore does not use the timeout."),
{ok, ConnPid} = gun:open("localhost", 12345, #{
event_handler => {gun_test_event_h, self()},
retry => 2,
retry_timeout => 1000
}),
{_, init, _} = receive_event(ConnPid),
%% Initial attempt.
{_, domain_lookup_start, _} = receive_event(ConnPid),
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _, ts := TS1}} = receive_event(ConnPid),
%% First retry is immediate.
{_, domain_lookup_start, #{ts := TS2}} = receive_event(ConnPid),
true = (TS2 - TS1) < 1000,
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _, ts := TS3}} = receive_event(ConnPid),
%% Second retry occurs after the retry_timeout.
{_, domain_lookup_start, #{ts := TS4}} = receive_event(ConnPid),
true = (TS4 - TS3) >= 1000,
{_, domain_lookup_end, _} = receive_event(ConnPid),
{_, connect_start, _} = receive_event(ConnPid),
{_, connect_end, #{error := _}} = receive_event(ConnPid),
{_, terminate, _} = receive_event(ConnPid),
ok.
server_name_indication_custom(_) ->
doc("Ensure a custom server_name_indication is accepted."),
do_server_name_indication("localhost", net_adm:localhost(), #{
tls_opts => [
{verify, verify_none}, {versions, ['tlsv1.2']},
{fail_if_no_peer_cert, false},
{server_name_indication, net_adm:localhost()}]
}).
server_name_indication_default(_) ->
doc("Ensure a default server_name_indication is accepted."),
do_server_name_indication(net_adm:localhost(), net_adm:localhost(), #{
tls_opts => [{verify, verify_none}, {versions, ['tlsv1.2']},
{fail_if_no_peer_cert, false}]
}).
do_server_name_indication(Host, Expected, GunOpts) ->
Self = self(),
{ok, OriginPid, OriginPort} = init_origin(tls, http,
fun(_, _, ClientSocket, _) ->
{ok, Info} = ssl:connection_information(ClientSocket),
Msg = {sni_hostname, _} = lists:keyfind(sni_hostname, 1, Info),
Self ! Msg
end),
{ok, ConnPid} = gun:open(Host, OriginPort, GunOpts#{
transport => tls,
retry => 0
}),
handshake_completed = receive_from(OriginPid),
%% The connection will succeed, look up the SNI hostname
%% and send it to us as a message, where we can check it.
{sni_hostname, Expected} = receive Msg = {sni_hostname, _} -> Msg end,
gun:close(ConnPid).
set_owner(_) ->
doc("The owner of the connection can be changed."),
Self = self(),
spawn(fun() ->
{ok, ConnPid} = gun:open("localhost", 12345),
gun:set_owner(ConnPid, Self),
Self ! {conn, ConnPid}
end),
ConnPid = receive {conn, C} -> C end,
#{owner := Self} = gun:info(ConnPid),
gun:close(ConnPid).
shutdown_reason(_) ->
doc("The last connection failure must be propagated."),
do_shutdown_reason().
do_shutdown_reason() ->
%% We set retry=1 so that we can monitor before the process terminates.
{ok, ConnPid} = gun:open("localhost", 12345, #{
retry => 1,
retry_timeout => 500
}),
Ref = monitor(process, ConnPid),
receive
%% Depending on timings we may monitor AFTER the process already
%% failed to connect and exited. In that case we just try again.
%% We rely on timetrap_timeout to stop the test if it takes too long.
{'DOWN', Ref, process, ConnPid, noproc} ->
ct:log("Monitor got noproc, trying again..."),
do_shutdown_reason();
{'DOWN', Ref, process, ConnPid, Reason} ->
{shutdown, econnrefused} = Reason,
gun:close(ConnPid)
end.
stream_info_http(_) ->
doc("Ensure the function gun:stream_info/2 works as expected for HTTP/1.1."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http,
fun(_, _, ClientSocket, ClientTransport) ->
%% Wait for the cancel signal.
receive cancel -> ok end,
%% Then terminate the stream.
ClientTransport:send(ClientSocket,
"HTTP/1.1 200 OK\r\n"
"content-length: 0\r\n"
"\r\n"
),
receive disconnect -> ok end
end),
{ok, Pid} = gun:open("localhost", OriginPort, #{
event_handler => {gun_test_event_h, self()}
}),
{ok, http} = gun:await_up(Pid),
{ok, undefined} = gun:stream_info(Pid, make_ref()),
StreamRef = gun:get(Pid, "/"),
Self = self(),
{ok, #{
ref := StreamRef,
reply_to := Self,
state := running
}} = gun:stream_info(Pid, StreamRef),
gun:cancel(Pid, StreamRef),
{ok, #{
ref := StreamRef,
reply_to := Self,
state := stopping
}} = gun:stream_info(Pid, StreamRef),
%% Cancel and wait for the stream to be canceled.
OriginPid ! cancel,
receive_event(Pid, cancel),
fun F() ->
case gun:stream_info(Pid, StreamRef) of
{ok, undefined} -> ok;
{ok, #{state := stopping}} -> F()
end
end(),
%% Wait for the connection to terminate.
OriginPid ! disconnect,
receive_event(Pid, disconnect),
{error, not_connected} = gun:stream_info(Pid, StreamRef),
gun:close(Pid).
stream_info_http2(_) ->
doc("Ensure the function gun:stream_info/2 works as expected for HTTP/2."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http2,
fun(_, _, _, _) -> receive disconnect -> ok end end),
{ok, Pid} = gun:open("localhost", OriginPort, #{
event_handler => {gun_test_event_h, self()},
protocols => [http2]
}),
{ok, http2} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
{ok, undefined} = gun:stream_info(Pid, make_ref()),
StreamRef = gun:get(Pid, "/"),
Self = self(),
{ok, #{
ref := StreamRef,
reply_to := Self,
state := running
}} = gun:stream_info(Pid, StreamRef),
gun:cancel(Pid, StreamRef),
%% Wait for the connection to terminate.
OriginPid ! disconnect,
receive_event(Pid, disconnect),
{error, not_connected} = gun:stream_info(Pid, StreamRef),
gun:close(Pid).
supervise_false(_) ->
doc("The supervise option allows starting without a supervisor."),
{ok, _, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort, #{supervise => false}),
{ok, http} = gun:await_up(Pid),
[] = [P || {_, P, _, _} <- supervisor:which_children(gun_sup), P =:= Pid],
ok.
tls_handshake_error_gun_http2_init_retry_0(_) ->
doc("Ensure an early TLS connection close is propagated "
"to the user of the connection."),
%% The server will immediately close the connection upon
%% establishment so that the client's socket is down
%% before it attempts to initialise HTTP/2.
%%
%% We use 'http' for the server to skip the HTTP/2 init
%% but the client is connecting using 'http2'.
{ok, _, OriginPort} = init_origin(tls, http,
fun(_, _, ClientSocket, ClientTransport) ->
ClientTransport:close(ClientSocket)
end),
{ok, ConnPid} = gun:open("localhost", OriginPort, #{
event_handler => {gun_test_fun_event_h, #{
tls_handshake_end => fun(_, #{socket := _}) ->
%% We sleep right after the gun_tls:connect succeeds to make
%% sure the next call to the socket fails (as the socket
%% should be disconnected at that point by the server because
%% we are not sending a certificate). The call we want to fail
%% is the sending of the HTTP/2 preface in gun_http2:init.
timer:sleep(1000)
end
}},
protocols => [http2],
retry => 0,
transport => tls,
tls_opts => [{verify, verify_none}]
}),
{error, {down, {shutdown, closed}}} = gun:await_up(ConnPid),
gun:close(ConnPid).
tls_handshake_error_gun_http2_init_retry_1(_) ->
doc("Ensure an early TLS connection close is propagated "
"to the user of the connection and does not result "
"in an infinite connect loop."),
%% The server will immediately close the connection upon
%% establishment so that the client's socket is down
%% before it attempts to initialise HTTP/2.
%%
%% We use 'http' for the server to skip the HTTP/2 init
%% but the client is connecting using 'http2'.
%%
%% We immediately accept another connection to handle
%% the coming retry and close that connection again.
{ok, _, OriginPort} = init_origin(tls, http,
fun(_, ListenSocket, ClientSocket1, ClientTransport) ->
ClientTransport:close(ClientSocket1),
%% We immediately accept a second connection and close it again.
{ok, ClientSocket2t} = ssl:transport_accept(ListenSocket, 5000),
{ok, ClientSocket2} = ssl:handshake(ClientSocket2t, 5000),
ClientTransport:close(ClientSocket2)
end),
{ok, ConnPid} = gun:open("localhost", OriginPort, #{
event_handler => {gun_test_fun_event_h, #{
tls_handshake_end => fun(_, #{socket := _}) ->
%% See tls_handshake_error_gun_http2_init_retry_0 for details.
timer:sleep(1000)
end
}},
protocols => [http2],
retry => 1,
transport => tls,
tls_opts => [{verify, verify_none}]
}),
{error, {down, {shutdown, closed}}} = gun:await_up(ConnPid),
gun:close(ConnPid).
tls_handshake_timeout(_) ->
doc("Ensure an integer value for tls_handshake_timeout is accepted."),
do_timeout(tls_handshake_timeout, 1000).
tls_handshake_timeout_infinity(_) ->
doc("Ensure infinity for tls_handshake_timeout is accepted."),
do_timeout(tls_handshake_timeout, infinity).
transform_header_name(_) ->
doc("The transform_header_name option allows changing the case of header names."),
{ok, ListenSocket} = gen_tcp:listen(0, [binary, {active, false}]),
{ok, {_, Port}} = inet:sockname(ListenSocket),
{ok, Pid} = gun:open("localhost", Port, #{
protocols => [http],
http_opts => #{
transform_header_name => fun(<<"host">>) -> <<"HOST">>; (N) -> N end
}
}),
{ok, ClientSocket} = gen_tcp:accept(ListenSocket, 5000),
{ok, http} = gun:await_up(Pid),
_ = gun:get(Pid, "/"),
{ok, Data} = gen_tcp:recv(ClientSocket, 0, 5000),
%% We do some very crude parsing of the response headers
%% to check that the header name was properly transformed.
Lines = binary:split(Data, <<"\r\n">>, [global]),
HostLines = [L || <<"HOST: ", _/bits>> = L <- Lines],
1 = length(HostLines),
gun:close(Pid).
unix_socket_connect(_) ->
case os:type() of
{win32, _} ->
{skip, "Unix Domain Sockets are not available on Windows."};
_ ->
do_unix_socket_connect()
end.
do_unix_socket_connect() ->
doc("Ensure we can send data via a unix domain socket."),
DataDir = "/tmp/gun",
SocketPath = filename:join(DataDir, "gun.sock"),
ok = filelib:ensure_dir(SocketPath),
_ = file:delete(SocketPath),
TCPOpts = [
{ifaddr, {local, SocketPath}},
binary, {nodelay, true}, {active, false},
{packet, raw}, {reuseaddr, true}
],
{ok, LSock} = gen_tcp:listen(0, TCPOpts),
Tester = self(),
Acceptor = fun() ->
{ok, S} = gen_tcp:accept(LSock),
{ok, R} = gen_tcp:recv(S, 0),
Tester ! {recv, R},
ok = gen_tcp:close(S),
ok = gen_tcp:close(LSock)
end,
spawn(Acceptor),
{ok, Pid} = gun:open_unix(SocketPath, #{}),
_ = gun:get(Pid, "/", [{<<"host">>, <<"localhost">>}]),
receive
{recv, _} ->
gun:close(Pid)
end.
uppercase_header_name(_) ->
doc("Header names may be given with uppercase characters."),
{ok, OriginPid, OriginPort} = init_origin(tcp, http),
{ok, Pid} = gun:open("localhost", OriginPort),
{ok, http} = gun:await_up(Pid),
handshake_completed = receive_from(OriginPid),
_ = gun:get(Pid, "/", [
{<<"USER-agent">>, "Gun/uppercase-headers"}
]),
Data = receive_from(OriginPid),
Lines = binary:split(Data, <<"\r\n">>, [global]),
[<<"user-agent: Gun/uppercase-headers">>] = [L || <<"user-agent: ", _/bits>> = L <- Lines],
gun:close(Pid).

22
gun/test/gun_ct_hook.erl Normal file
View File

@ -0,0 +1,22 @@
%% Copyright (c) 2015-2023, Loïc Hoguin <essen@ninenines.eu>
%%
%% Permission to use, copy, modify, and/or distribute this software for any
%% purpose with or without fee is hereby granted, provided that the above
%% copyright notice and this permission notice appear in all copies.
%%
%% THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
%% WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
%% MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
%% ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
%% WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
%% ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
%% OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-module(gun_ct_hook).
-export([init/2]).
init(_, _) ->
ct_helper:start([cowboy, gun]),
ct_helper:make_certs_in_ets(),
{ok, undefined}.

Some files were not shown because too many files have changed in this diff Show More