Rand Stats

HTTP::Tiny

zef:jjatria

NAME

HTTP::Tiny - A small, simple, correct HTTP/1.1 client

SYNOPSIS

use HTTP::Tiny;

my $response = HTTP::Tiny.new.get: 'http://httpbin.org/get';

die "Failed!\n" unless $response<success>;

say "$response<status> $response<reason>";
# OUTPUT:
# 200 OK

for $response<headers>.kv -> $key, $v {
    for $v.List -> $value {
        say "$key: $value";
    }
}
# OUTPUT:
# content-type: application/json
# date: Fri, 09 Oct 2020 21:49:38 GMT
# connection: close
# content-length: 230
# server: gunicorn/19.9.0
# access-control-allow-origin: *
# access-control-allow-credentials: true

print $response<content>.decode if $response<content>;
# OUTPUT:
# {
#   "args": {},
#   "headers": {
#     "Host": "httpbin.org",
#     "User-Agent": "HTTP-Tiny",
#     "X-Amzn-Trace-Id": "..."
#   },
#   "origin": "...",
#   "url": "http://httpbin.org/get"
# }

DESCRIPTION

This is a very simple but correct HTTP/1.1 client, designed for doing simple requests without the overhead of a large framework like HTTP::UserAgent.

It is a Raku port of the Perl library of the same name. It supports redirection, streaming requests and responses, multipart and URL-encoded form uploads, and correctly handles multipart responses to ranged requests.

Cookie support depends on external classes, such as Cookie::Jar or some other class with a compatible interface.

METHODS

Calling the new method to construct an object is optional when using the methods described in this section. When not doing so, new will be called automatically before executing the request, and the created object will be discarded after the request is complete.

new

method new (
    Str   :$agent = 'HTTP-Tiny/VERSION Raku',
          :$cookie-jar,
          :%default-headers,
    Str   :$http-proxy,
    Str   :$https-proxy,
    Bool  :$keep-alive = True,
    Int   :$max-redirect = 5,
    Set() :%no-proxy,
    Bool  :$throw-exceptions,
) returns HTTP::Tiny

Creates a new HTTP::Tiny object. The following attributes are parameters:

agent

A Str to use as the value of the User-Agent header. Defaults to 'HTTP-Tiny/$VERSION Raku'.

An object to use for parsing and storing cookies and generating the Cookie header. The object must have an add method that can be called with the URL and a Set-Cookie string to store a cookie; and a header method that can be called with the request method and URL to generate the appropriate Cookie header.

See Cookie::Jar for one possible implementation of such a cookie jar class.

default-headers

A Hash of default headers to apply to requests. Headers specified during the call take precedence over the ones specified here.

max-redirect

Maximum number of redirects allowed. Defaults to 5. Set to 0 to prevent redirection.

keep-alive

Whether to re-use the last connection, if it is for the same scheme, host, and port. Defaults to True.

throw-exceptions

When set to True, non-success HTTP responses will throw a X::HTTP::Tiny exception. The original error response Hash will be available as the result of the .response method of the exception.

proxy

URL of a generic proxy server for both HTTP and HTTPS connections.

Defaults to the value in the all_proxy or ALL_PROXY environment variables (in that order). Set to the empty string to ignore variables set in the environment.

no-proxy

Set of domain suffixes that should not be proxied. Any value that implements the Set method is allowed. A Str is also allowed, in which case it must be a comma-separated list of suffixes that will be split, trimmed, and coerced to a Set.

Defaults to the value in the no_proxy, which will be treated like the Str case described above.

http-proxy

URL of a proxy server for HTTP connections.

Defaults to the value in the http_proxy or HTTP_PROXY environment variables, or to the value of the proxy parameter described above (in that order). Set to the empty string to ignore variables set in the environment.

https-proxy

URL of a proxy server for HTTPS connections.

Defaults to the value in the https_proxy or HTTPS_PROXY environment variables, or to the value of the proxy parameter described above (in that order). Set to the empty string to ignore variables set in the environment.

delete

Shorthand method for calling request with 'DELETE' as the method. See the documentation for request for full details on the supported parameters and the return value.

get

Shorthand method for calling request with 'GET' as the method. See the documentation for request for full details on the supported parameters and the return value.

head

Shorthand method for calling request with 'HEAD' as the method. See the documentation for request for full details on the supported parameters and the return value.

options

Shorthand method for calling request with 'OPTIONS' as the method. See the documentation for request for full details on the supported parameters and the return value.

patch

Shorthand method for calling request with 'PATCH' as the method. See the documentation for request for full details on the supported parameters and the return value.

post

Shorthand method for calling request with 'POST' as the method. See the documentation for request for full details on the supported parameters and the return value.

put

Shorthand method for calling request with 'PUT' as the method. See the documentation for request for full details on the supported parameters and the return value.

trace

Shorthand method for calling request with 'TRACE' as the method. See the documentation for request for full details on the supported parameters and the return value.

request

method request (
    Str $method,
    Str $url,
       :%headers,
       :$content,
       :&data-callback,
       :&trailer-callback,
) returns Hash

Executes an HTTP request of the given method type on the given URL. The URL must have unsafe characters escaped and international domains encoded. Valid HTTP methods are 'GET', 'DELETE', 'HEAD', 'OPTIONS', 'PATCH', 'POST', 'PUT', and 'TRACE', with their names being case sensitive as per the HTTP/1.1 specification.

If the URL includes a "user:password" stanza, they will be used for Basic-style authorisation headers. For example:

$ua.request: 'GET', 'http://Aladdin:open sesame@example.com/';

If the "user:password" stanza contains reserved characters, they must be percent-escaped:

$ua.request: 'GET', 'http://john%40example.com:password@example.com/';

The Authorization header generated from these data will not be included in a redirected request. If you want to avoid this behaviour you can set the value manually, in which case it will not be modified or ignored.

The remaining named parameters are detailed below.

%headers

A map of headers to include with the request. If the value is a List of strings, the header will be output multiple times, once with each value in the array. The headers specified in this parameter overwrite any default ones.

The Host header is internally generated from the request URL in accordance with RFC 7230. It is a fatal error to specify this header. Other headers may be ignored or overwritten if necessary for transport compliance, but this will in general be avoided.

$content

A value to use for the body of the request, which can be a Blob, a Str or Numeric, a Hash, or a Callable, with each of these modifying the default assumptions about the request.

If $content is a Blob, the Content-Type header will default to application/octet-stream and the contents of the Blob will be used as-is as the body. The Content-Length header will also default to the number of bytes in the Blob.

If $content is a Str or Numeric, it will be stringified by calling Str on it and internally encoded as UTF-8 and converted to a Blob. The Content-Type will in this case default to text/plain;charset=UTF-8, but handling will otherwise be as detailed above.

If $content is a Hash, the default content type will depend on the values. If any of the values is an IO::Path object it will be multipart/form-data, otherwise it will be application/x-www-form-urlencoded.

If $content is a Callable, it will be called iteratively to produce the body of the request. When called, it must return a Blob with the next part of the body until the body has been fully generated, in which case it must return an empty Blob, or a Blob type object.

Note that these behaviours are the default behaviours, and represent the assumptions that will be made about the request based on the input.

When using a Callable, the Content-Type will default to application/octet-stream and if no Content-Length header has been set, the Transfer-Encoding will default to chunked, with each new part of the body being sent in a separate chunk.

When using a Hash, its contents will be encoded depending on the value of the Content-Type header. Using IO::Path objects as values is only supported with multipart form encoding. If a value is a IO::Path, IO::Handle, or anything that supports the slurp method, this will be called with the :bin argument to provide the value of that key, and the content type will be set to application/octet-stream. If using an IO::Path object, the filename will be set to the result of calling basename.

If no value is set, no Content-Type or Content-Length headers will be generated.

&data-callback

The data callback takes a block of code that will be executed once with each chunk of the response body. The callback will be introspected to determine how many arguments it can receive, and will be called with up to three arguments each time:

This should allow customising the behaviour of the callback depending on the response status or headers before receiving the full response body.

The callback must support at least the Blob argument. The other two are optional. Not supporting any of these is an error.

&trailer-callback

When using a chunked transfer encoding, this callback will be called once after the request body has been sent. It should return a Hash which will be used to add trailing headers to the request.

The response Hash

The request method returns a Hash with the response. The Hash will have the following keys:

success : A Bool that will be true if the response status code starts with a 2.

url : The URL that provided the response as a Str. This will be the URL provided by the caller unless there were redirections, in which case it will be the last URL queried in the redirection chain.

status : The HTTP status code of the response as an Int.

reason : The response phrase as provided by the server.

content : The body of the response as a Buf[uint8]. This key will be missing if the response had no content or if a data callback was provided to consume the body. HTTP::Tiny will never automatically decode a response body.

headers : A Hash of header fields. All header fields will be normalised to be lower case. If a header is repeated, the value will be a List with the received values as Str objects. Otherwise, the value will be a Str. Header values will be decoded using ISO-8859-1 as per RFC 7230 § 3.2.4.

protocol : The protocol of the response, such as 'HTTP/1.1' or 'HTTP/1.0'.

redirects : If this key exists, it will hold a List of response Hash objects from the encountered redirects in the order they occurred. This key will no exist if no redirection took place.

If an exception is encountered during execution, the status field will be set to '599' and the content field will hold the text of the exception.

mirror

method mirror (
    Str  $url,
    IO() $file,
        :$content,
        :%headers,
        :&trailer-callback,
) returns Hash

Executes a GET request for the URL and saves the response body to the specified file. The URL must have unsafe characters escaped and international domain names encoded. If the file already exists, the request will include an If-Modified-Since header with the modification timestamp of the file if none has already been provided in the :%headers parameter. The parent directories of the file will not be automatically created.

The value of $file can be anything that implements an .IO method.

The success field of the response will be true if the status code is 2XX or if the status code is 304 (unmodified).

If the file was modified and the server response includes a properly formatted Last-Modified header, the file modification time will be updated accordingly. Note that currently this makes use of the touch system command,and will therefore not work if this command is not available.

can-ssl

with HTTP::Tiny.can-ssl {
    # SSL support is available
}
else {
    note 'SSL support not available: ' ~ .exception.message;
}

Indicates if SSL support is available by checking for the correct version of IO::Socket::SSL (greater than or equal to 0.0.2). It will either return True if SSL support is available, or a Failure indicating why it isn't.

PROXY SUPPORT

HTTP::Tiny can proxy both HTTP and HTTPS requests. Only Basic proxy authorization is supported and it must be provided as part of the proxy URL, as in http://user:pass@proxy.example.com/.

HTTP::Tiny supports the following proxy environment variables:

An HTTPS connection may be made via an HTTP proxy that supports the CONNECT method (cf. RFC 2817). If your proxy itself uses HTTPS, you can not tunnel HTTPS over it.

Be warned that proxying an HTTPS connection opens you to the risk of a man-in-the-middle attack by the proxy server.

The no_proxy environment variable is supported in the format of a comma-separated list of domain extensions proxy should not be used for.

Proxy arguments passed to new will override their corresponding environment variables.

LIMITATIONS

HTTP::Tiny aims to be conditionally compliant with the HTTP/1.1 specifications:

It aims to meet all "MUST" requirements of the specification, but only some of the "SHOULD" requirements.

Some particular limitations of note include:

SEE ALSO

Other Clients

Cro::HTTP

Part of the Cro family of libraries, it is written with asynchronous code as its primary goal. Supports HTTP/2.0.

Curlie

A recent high-level interface based on libcurl. Depends on LibCurl, mentioned below.

HTTP::Tinyish

Another port from Perl, HTTP::Tinyish offers a similar interface to this library while relying on an externally available curl binary.

HTTP::UserAgent

The de-facto blocking HTTP client for Raku, used by most applications. If a feature you want is not supported by HTTP::Tiny, try using this distribution. It is included in the Rakudo Star distribution, so chances are you already have it.

That said, at the time of writing HTTP::UserAgent does not handle 1XX responses, nor does it support chunked requests.

LibCurl

Raku bindings for libcurl. The bindings are fairly low-level, so they allow for finer control than HTTP::Tinyish, but at the cost of a more complicated interface.

LWP::Simple

An older an more barebones blocking HTTP client for Raku, preceding the development of HTTP::UserAgent.

Net::HTTP

A library providing the building blocks to write your own HTTP client. Supports connection caching and should be thread safe.

Code is fairly low-level, so use in real-world scenarios might require some effort until more progress is done in the implementation of classes like Net::HTTP::Client.

WWW

This distribution exports a set of routines to run HTTP requests. It does not implement its own transport layer, but is instead built on top of HTTP::UserAgent.

Additional Features

Cookie::Jar

Required for cookie support. Any class that supports add and header methods like the ones in this class can be used with HTTP::Tiny.

IO::Socket::SSL

Required for HTTPS support.

AUTHOR

José Joaquín Atria jjatria@cpan.org

ACKNOWLEDGEMENTS

The code in this distribution is heavily inspired by that of the Perl library of the same name, written by Christian Hansen and David Golden.

Some parts of the code have been adapted from existing solutions in the HTTP::UserAgent codebase, which served as a reference on the use of the Raku toolbox.

Copyright 2019-2022 José Joaquín Atria

This library is free software; you can redistribute it and/or modify it under the Artistic License 2.0.