NAME
HTTP::Tiny - A small, simple, correct HTTP/1.1 client
SYNOPSIS
use HTTP::Tiny;
my $response = HTTP::Tiny.new.get: 'http://httpbin.org/get';
die "Failed!\n" unless $response<success>;
say "$response<status> $response<reason>";
# OUTPUT:
# 200 OK
for $response<headers>.kv -> $key, $v {
for $v.List -> $value {
say "$key: $value";
}
}
# OUTPUT:
# content-type: application/json
# date: Fri, 09 Oct 2020 21:49:38 GMT
# connection: close
# content-length: 230
# server: gunicorn/19.9.0
# access-control-allow-origin: *
# access-control-allow-credentials: true
print $response<content>.decode if $response<content>;
# OUTPUT:
# {
# "args": {},
# "headers": {
# "Host": "httpbin.org",
# "User-Agent": "HTTP-Tiny",
# "X-Amzn-Trace-Id": "..."
# },
# "origin": "...",
# "url": "http://httpbin.org/get"
# }
DESCRIPTION
This is a very simple but correct HTTP/1.1 client, designed for doing simple
requests without the overhead of a large framework like HTTP::UserAgent.
It is a Raku port of the Perl library of the same name. It supports
redirection, streaming requests and responses, multipart and URL-encoded
form uploads, and correctly handles multipart responses to ranged requests.
Cookie support depends on external classes, such as Cookie::Jar or some
other class with a compatible interface.
METHODS
Calling the new method to construct an object is optional when using the
methods described in this section. When not doing so, new will be called
automatically before executing the request, and the created object will be
discarded after the request is complete.
new
method new (
Str :$agent = 'HTTP-Tiny/VERSION Raku',
:$cookie-jar,
:%default-headers,
Str :$http-proxy,
Str :$https-proxy,
Bool :$keep-alive = True,
Int :$max-redirect = 5,
Set() :%no-proxy,
Bool :$throw-exceptions,
) returns HTTP::Tiny
Creates a new HTTP::Tiny object. The following attributes are parameters:
agent
A Str to use as the value of the User-Agent header. Defaults to
'HTTP-Tiny/$VERSION Raku'.
cookie-jar
An object to use for parsing and storing cookies and generating the Cookie
header. The object must have an add method that can be called with the URL
and a Set-Cookie string to store a cookie; and a header method that can
be called with the request method and URL to generate the appropriate
Cookie header.
See Cookie::Jar for one possible implementation of such a cookie jar class.
A Hash of default headers to apply to requests. Headers specified during the
call take precedence over the ones specified here.
max-redirect
Maximum number of redirects allowed. Defaults to 5. Set to 0 to prevent
redirection.
keep-alive
Whether to re-use the last connection, if it is for the same scheme, host,
and port. Defaults to True.
throw-exceptions
When set to True, non-success HTTP responses will throw a X::HTTP::Tiny
exception. The original error response Hash will be available as the result of
the .response method of the exception.
proxy
URL of a generic proxy server for both HTTP and HTTPS connections.
Defaults to the value in the all_proxy or ALL_PROXY environment
variables (in that order). Set to the empty string to ignore variables set in
the environment.
no-proxy
Set of domain suffixes that should not be proxied. Any value that implements
the Set method is allowed. A Str is also allowed, in which case it must be
a comma-separated list of suffixes that will be split, trimmed, and coerced to
a Set.
Defaults to the value in the no_proxy, which will be treated like the Str
case described above.
http-proxy
URL of a proxy server for HTTP connections.
Defaults to the value in the http_proxy or HTTP_PROXY environment
variables, or to the value of the proxy parameter described above (in that
order). Set to the empty string to ignore variables set in the environment.
https-proxy
URL of a proxy server for HTTPS connections.
Defaults to the value in the https_proxy or HTTPS_PROXY environment
variables, or to the value of the proxy parameter described above (in that
order). Set to the empty string to ignore variables set in the environment.
delete
Shorthand method for calling request with 'DELETE' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
get
Shorthand method for calling request with 'GET' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
head
Shorthand method for calling request with 'HEAD' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
options
Shorthand method for calling request with 'OPTIONS' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
patch
Shorthand method for calling request with 'PATCH' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
post
Shorthand method for calling request with 'POST' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
put
Shorthand method for calling request with 'PUT' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
trace
Shorthand method for calling request with 'TRACE' as the method. See the
documentation for request for full details on the supported parameters and
the return value.
request
method request (
Str $method,
Str $url,
:%headers,
:$content,
:&data-callback,
:&trailer-callback,
) returns Hash
Executes an HTTP request of the given method type on the given URL. The URL
must have unsafe characters escaped and international domains encoded. Valid
HTTP methods are 'GET', 'DELETE', 'HEAD', 'OPTIONS', 'PATCH', 'POST', 'PUT',
and 'TRACE', with their names being case sensitive as per the HTTP/1.1
specification.
If the URL includes a "user:password" stanza, they will be used for
Basic-style authorisation headers. For example:
$ua.request: 'GET', 'http://Aladdin:open sesame@example.com/';
If the "user:password" stanza contains reserved characters, they must
be percent-escaped:
$ua.request: 'GET', 'http://john%40example.com:password@example.com/';
The Authorization header generated from these data will not be included
in a redirected request. If you want to avoid this behaviour you can set the
value manually, in which case it will not be modified or ignored.
The remaining named parameters are detailed below.
A map of headers to include with the request. If the value is a List of
strings, the header will be output multiple times, once with each value in the
array. The headers specified in this parameter overwrite any default ones.
The Host header is internally generated from the request URL in accordance
with RFC 7230. It is a fatal error to specify this header. Other
headers may be ignored or overwritten if necessary for transport compliance,
but this will in general be avoided.
$content
A value to use for the body of the request, which can be a Blob, a Str or
Numeric, a Hash, or a Callable, with each of these modifying the default
assumptions about the request.
If $content is a Blob, the Content-Type header will default to
application/octet-stream and the contents of the Blob will be used as-is as
the body. The Content-Length header will also default to the number of
bytes in the Blob.
If $content is a Str or Numeric, it will be stringified by calling Str
on it and internally encoded as UTF-8 and converted to a Blob. The
Content-Type will in this case default to text/plain;charset=UTF-8,
but handling will otherwise be as detailed above.
If $content is a Hash, the default content type will depend on the values.
If any of the values is an IO::Path object it will be multipart/form-data,
otherwise it will be application/x-www-form-urlencoded.
If $content is a Callable, it will be called iteratively to produce the
body of the request. When called, it must return a Blob with the next part
of the body until the body has been fully generated, in which case it must
return an empty Blob, or a Blob type object.
Note that these behaviours are the default behaviours, and represent the
assumptions that will be made about the request based on the input.
When using a Callable, the Content-Type will default to
application/octet-stream and if no Content-Length header has been set,
the Transfer-Encoding will default to chunked, with each new part of the
body being sent in a separate chunk.
When using a Hash, its contents will be encoded depending on the value of the
Content-Type header. Using IO::Path objects as values is only supported
with multipart form encoding. If a value is a IO::Path, IO::Handle, or
anything that supports the slurp method, this will be called with the
:bin argument to provide the value of that key, and the content type will
be set to application/octet-stream. If using an IO::Path object, the
filename will be set to the result of calling basename.
If no value is set, no Content-Type or Content-Length headers will be
generated.
&data-callback
The data callback takes a block of code that will be executed once with each
chunk of the response body. The callback will be introspected to determine
how many arguments it can receive, and will be called with up to three
arguments each time:
A Blob with the current encoded response chunk
A Hash with the current state of the response Hash
A Hash with the part headers (only for multipart responses)
This should allow customising the behaviour of the callback depending on the
response status or headers before receiving the full response body.
The callback must support at least the Blob argument. The other two are
optional. Not supporting any of these is an error.
&trailer-callback
When using a chunked transfer encoding, this callback will be called once after
the request body has been sent. It should return a Hash which will be used to
add trailing headers to the request.
The response Hash
The request method returns a Hash with the response. The Hash will have the
following keys:
success
: A Bool that will be true if the response status code starts with a 2.
url
: The URL that provided the response as a Str. This will be the URL provided by
the caller unless there were redirections, in which case it will be the last
URL queried in the redirection chain.
status
: The HTTP status code of the response as an Int.
reason
: The response phrase as provided by the server.
content
: The body of the response as a Buf[uint8]. This key will be missing if the
response had no content or if a data callback was provided to consume the
body. HTTP::Tiny will never automatically decode a response body.
headers
: A Hash of header fields. All header fields will be normalised to be lower
case. If a header is repeated, the value will be a List with the received
values as Str objects. Otherwise, the value will be a Str. Header values will
be decoded using ISO-8859-1 as per RFC 7230 § 3.2.4.
protocol
: The protocol of the response, such as 'HTTP/1.1' or 'HTTP/1.0'.
redirects
: If this key exists, it will hold a List of response Hash objects from the
encountered redirects in the order they occurred. This key will no exist if
no redirection took place.
If an exception is encountered during execution, the status field will be
set to '599' and the content field will hold the text of the exception.
mirror
method mirror (
Str $url,
IO() $file,
:$content,
:%headers,
:&trailer-callback,
) returns Hash
Executes a GET request for the URL and saves the response body to the
specified file. The URL must have unsafe characters escaped and
international domain names encoded. If the file already exists, the request
will include an If-Modified-Since header with the modification timestamp
of the file if none has already been provided in the :%headers parameter.
The parent directories of the file will not be automatically created.
The value of $file can be anything that implements an .IO method.
The success field of the response will be true if the status code is 2XX
or if the status code is 304 (unmodified).
If the file was modified and the server response includes a properly formatted
Last-Modified header, the file modification time will be updated
accordingly. Note that currently this makes use of the touch system
command,and will therefore not work if this command is not available.
can-ssl
with HTTP::Tiny.can-ssl {
# SSL support is available
}
else {
note 'SSL support not available: ' ~ .exception.message;
}
Indicates if SSL support is available by checking for the correct version
of IO::Socket::SSL (greater than or equal to 0.0.2). It will either return
True if SSL support is available, or a Failure indicating why it isn't.
PROXY SUPPORT
HTTP::Tiny can proxy both HTTP and HTTPS requests. Only Basic proxy
authorization is supported and it must be provided as part of the proxy URL,
as in http://user:pass@proxy.example.com/.
HTTP::Tiny supports the following proxy environment variables:
An HTTPS connection may be made via an HTTP proxy that supports the
CONNECT method (cf. RFC 2817). If your proxy itself uses HTTPS, you can
not tunnel HTTPS over it.
Be warned that proxying an HTTPS connection opens you to the risk of a
man-in-the-middle attack by the proxy server.
The no_proxy environment variable is supported in the format of a
comma-separated list of domain extensions proxy should not be used for.
Proxy arguments passed to new will override their corresponding
environment variables.
LIMITATIONS
HTTP::Tiny aims to be conditionally compliant with the
HTTP/1.1 specifications:
It aims to meet all "MUST" requirements of the specification, but only some
of the "SHOULD" requirements.
Some particular limitations of note include:
HTTP::Tiny focuses on correct transport. Users are responsible for
ensuring that user-defined headers and content are compliant with the
HTTP/1.1 specification.
Users must ensure that URLs are properly escaped for unsafe characters
and that international domain names are properly encoded to ASCII.
Redirection is very strict against the specification. Redirection is
only automatic for response codes 301, 302, 307 and 308 if the request
method is GET or HEAD. Response code 303 is always converted into a
GET redirection, as mandated by the specification. There is no
automatic support for status 305 ("Use proxy") redirections.
In the event of a redirect, the content of the original request will
not be included, and any headers that make reference to the content will
be removed. This also applies to other headers that do not make reference
to content, but that are likewise rendered stale by the redirect, as per
RFC 9110 § 15.4.
There is no provision for delaying a request body using an Expect
header. Unexpected 1XX responses are silently ignored as per the
specification.
Only 'chunked' Transfer-Encoding is supported.
There is no support for a Request-URI of * for the OPTIONS request.
Headers mentioned in the RFCs and some other, well-known headers are
generated with their canonical case. The order of headers is not
preserved: control headers are sent first, while the remainder are sent
in an unspecified order.
No mitigations for httpoxy have been implemented.
If you are using this library under CGI, you are on your own.
SEE ALSO
Other Clients
Part of the Cro family of libraries, it is written with asynchronous code as
its primary goal. Supports HTTP/2.0.
A recent high-level interface based on libcurl. Depends on LibCurl,
mentioned below.
Another port from Perl, HTTP::Tinyish offers a similar interface to this
library while relying on an externally available curl binary.
The de-facto blocking HTTP client for Raku, used by most applications. If a
feature you want is not supported by HTTP::Tiny, try using this distribution.
It is included in the Rakudo Star distribution, so chances are you already
have it.
That said, at the time of writing HTTP::UserAgent does not handle 1XX
responses, nor does it support chunked requests.
Raku bindings for libcurl. The bindings are fairly low-level, so they allow
for finer control than HTTP::Tinyish, but at the cost of a more complicated
interface.
An older an more barebones blocking HTTP client for Raku, preceding the
development of HTTP::UserAgent.
A library providing the building blocks to write your own HTTP client.
Supports connection caching and should be thread safe.
Code is fairly low-level, so use in real-world scenarios might require
some effort until more progress is done in the implementation of classes
like Net::HTTP::Client.
This distribution exports a set of routines to run HTTP requests. It does
not implement its own transport layer, but is instead built on top of
HTTP::UserAgent.
Additional Features
Required for cookie support. Any class that supports add and header
methods like the ones in this class can be used with HTTP::Tiny.
Required for HTTPS support.
AUTHOR
José Joaquín Atria jjatria@cpan.org
ACKNOWLEDGEMENTS
The code in this distribution is heavily inspired by that of
the Perl library of the same name,
written by Christian Hansen and David Golden.
Some parts of the code have been adapted from existing solutions in the
HTTP::UserAgent codebase, which served as a reference on the use of the
Raku toolbox.
COPYRIGHT AND LICENSE
Copyright 2019-2022 José Joaquín Atria
This library is free software; you can redistribute it and/or modify it
under the Artistic License 2.0.