webmockr
) async requests via
either Async
and/or AsyncVaried
(thanks @keller-mark)
(#180)_PACKAGE
pattern for pkg level man file
(#177)Via the two issues (#135) and (#159) crul
now supports
http retries in: Async
, AsyncVaried
,
AsyncQueue
, and the request builder class
HttpRequest
Async
when urls
and disk
vector lengths do not match (they must match)
(#165) thanks @shakrazAsyncQueue
docs (#146) thanks @johnbaums !HttpClient
docs to state that it’s an R6 class,
and give some details on what an R6 class is and links to more info
(#155)AsyncQueue
gains methods: parse
,
status_code
, status
, content
, and
times
(#156)$responses()
method now returns an S3 class with an
associated print method to prevent printing a lot of results to the
screen; print method pritns a summary of results, and at most 10
results, just status code and url (#157)Paginator
gains support for query parameter combination
page
/per_page
to automatically paginate
(#145)Paginator
as well as for
HttpClient
(#151) (#152) (#153) thanks @ateuchercrul
would raise an error
when this header parsing happens, but now we raise a warning instead
(#150)ok()
can now accept more than 1 status code so that you
can check if the status of a url is within a set of status codes rather
than equal to 1 status code (#124)ok()
gains a parameter verb
to use either
head or get requests. in addition added more documentation (#125) to the
function on how to get the “right answer” for whether a url is ok/up
(#123) (#127)ok()
gains parameter ua_random
, which if
TRUE
, will use a random user agent string pulled from a
vector of 50 user agent strings generated from
charlatan::UserAgentProvider
(#138)AsyncQueue
for doing async
requests with rate limits (#139)curl_verbose()
and
set_verbose()
. curl_verbose()
can be set by
passing to the initialize step (e.g.,
HttpClient$new(url, verbose=curl_verbose())
), and gets more
compact verbose curl output, while also getting request body information
(and response body optionally). set_verbose()
is sets
curl_verbose()
globally (#141)HttpClient
vs. Async
)
(#133) (#143)AsyncVaried
class (#140)HttpRequest
now adds the query (if present) to the
printed url in the print method for the class (it was absent before now)
(#128)delete-requests
and post-requests
manual files - mostly redundant with other documentationHttpResponse
response object gains new methods for
checking response content types, includes: raise_for_ct
,
raise_for_ct_html
, raise_for_ct_json
,
raise_for_ct_xml
. these behave similarly to
raise_for_status
, and can behave as a warning or raise an
error through stop (#119) (#120)as.character.form_file
when both
httr and crul are loaded (#112)head()
verb on HttpClient
was no
capturing auth
when set on initialization (#122)jsonlite
package moved to Imports (#112)parse()
method in the HttpResponse
object now checks whether the response raw bytes can be converted to
character, and if not just returns raw bytes (#115) (#116)cainfo
curl option, fixes problem
arising from change in recent libcurl version (#117)parse()
method of an HttpResponse
class to the internally called
function iconv()
to finely control the usage of
iconv
for cases in which normal encoding conversion doesn’t
work (#110)httpcode
package instead of fauxpas
package within ok()
function (#108) (#109) thanks @maelle !HttpClient
. both request and
response hooks supported. not supported in async methods for now (#76)
(#107)$parse()
behavior (in the
HttpResponse
object) when using disk or stream.
$parse()
was throwing a warning when using disk and an
error when using stream. and improves behavior when doing async requests
(#104)Paginator
gains optional progress bar through the new
progress
parameter. In addition, the cat()
calls inside the method were removed, so as not to insert newlines with
each page and to not print “OK” when done (#106) thanks @boshekAsync
(#101)
(#103)AsyncVaried
with
curl
of a certain version, works now (#102) (#103)HttpClient
gains a retry
method: retries
any request verb until successful (HTTP response status < 400) or a
condition for giving up is met. (#89) (#95) thanks @hlappHttpClient
, HttpRequest
, and
Async
classes gain verb
method for doing HTTP
requests specifying any of the supported HTTP verbs (#97)HttpClient
and Paginator
gain a
url_fetch
method: get the URL that would be sent in an HTTP
request without sending the HTTP request. Useful for getting the URL
before executing an HTTP request if you need to check something about
the URL first. (#92)HttpResponse
class as
$response_headers_all
as an unnamed list, with each element
a named list of headers; the last list in the set is the final response
headers that match those given in the $response_headers
slot (#60) (#99)url_parse
: lacked check that input was a string,
and that it was length 1 - this PR fixed that (#100) thanks @aaronwolenHttpStubbedResponse
was removed from the package - it
may have been used at some point, but is not used in the package anymore
(#88)Async
and AsyncVaried
now support simple
auth, see ?auth
(#70)ok()
to ping a URL to see if it’s up
or not, returns a single boolean (#71) (#73)HttpClient
and HttpRequest
gain new
parameter progress
that accepts a function to use to
construct a progress bar. For now accepts httr::progress()
but will accept other options in the future (#20) (#81)set_auth()
, set_headers()
,
set_opts()
, set_proxy()
, and
crul_settings()
(#48) (#85)httpcode::http_code
(#80)Async
and
AsyncVaried
to print max of 10 and tell user how many total
and remaining not shown (#72)proxy()
for socks, e.g. to use with
TOR (#79)Async
and AsyncVaried
requests
fail, they don’t error but instead we capture the error and pass it back
in the result. this way any failure requests don’t stop progress of the
entire async request suite (#74) (#84)Paginator
- it wasn’t handling pagination
correctly. In addition, fixed to hopefully handle all scenarios now.
added more tests (#62)urltools::url_encode
to encode strings, but it wasn’t
encoding correctly in some locales. Using curl::curl_escape
fixes the problem. Encoding is done on query values and names (#67)
(#68)Paginator
to help users
automatically paginate through multiple requests. It only supports query
parameter based paginating for now. We’ll add support later for other
types including cursors (e.g., used in Solr servers), and for link
headers (e.g., used in the GitHub API). Please get in touch if you find
any problems with Paginator
. (#56)Async
and Asyncvaried
gain
ability to write to disk and stream data (to disk or elsewhere, e.g. R
console or to an R object) (#46) thanks @artemklevtsov for the push to do
thisauth
to indicate that
user
and pwd
are indeed required - and to
further indicate that one can pass in NULL
to those
parameters (similar to an empty string ""
in
httr::authenticate
) when one e.g. may want to use
gssnegotiate
method (#43)I()
(#55)head
requests with HttpClient
when passing query
parameter - it was failing previously.
Added query
parameter back. (#52)upload()
and
examples (#25)crul
now drops any options passed in to
opts
or to ...
that are not in set of allowed
curl options, see curl::curl_options()
(#49)?cookies
for how to set cookies
(#44)disk
and stream
from
head
method in HttpClient
and
HttpRequest
as no body returned in a HEAD requestAsyncVaried
to return async responses in the
order that they were passed in. This also fixes this exact same behavior
in Async
because Async
uses
AsyncVaried
internally. (#41) thanks @dirkschumacher for reportingwebmockr
, which is now on CRAN.auth()
to do simple authentication
(#33)HttpStubbedResponse
for making a stubbed
response object for the webmockr
integration (#4)mock()
to turn on mocking - it’s off by
default. If webmockr
is not installed but user attempts to
use mocking we error with message to install webmockr
(#4)gzip-deflate
by deafult for each request to make
sure gzip compression is used if the server can do it (#34)useragent
to User-Agent
as default
user agent header (#35)url_build
and
url_parse
(#31)AsyncVaried
(#30)crul
in realistic scenarios
rather than brief examples to demonstrate individual features (#29)HttpRequest
(#28)HttpClient
(#26)Async
and AsyncVaried
. The former
being a simpler interface treating all URLs with same options/HTTP
method, and the latter allowing any type of request through the new R6
class HttpRequest
(#8) (#24)HttpRequest
to support
AsyncVaried
- this method only defines a request, but does
not execute it. (#8)...
. And we check that user
doesn’t pass in prohibited options (curl
package takes care
of checking that options are valid) (#5)fauxpas
package for dealing with HTTP
conditions. It’s a Suggest, so only used if installed (#6)curl::curl_fetch_stream
. stream
param defaults
to NULL
(thus ignored), or pass in a function to use
streaming. Only one of memory, streaming or disk allowed. (#9)curl::curl_fetch_disk
.
disk
param defaults to NULL
(thus ignored), or
pass in a path to write to disk instead of use memory. Only one of
memory, streaming or disk allowed. (#12)raise_for_status()
method on the
HttpResponse
class (#10)httpcode
but wasn’t using it in the
package. Now using the package in HttpResponse