Object
The HTTPClient class provides several methods for accessing Web resources via HTTP.
HTTPClient instance is designed to be MT-safe. You can call a HTTPClient instance from several threads without synchronization after setting up an instance.
clnt = HTTPClient.new clnt.set_cookie_store('/home/nahi/cookie.dat') urls.each do |url| Thread.new(url) do |u| p clnt.head(u).status end end
At first, how to create your client. See initialize for more detail.
Create simple client.
clnt = HTTPClient.new
Accessing resources through HTTP proxy. You can use environment variable 'http_proxy' or 'HTTP_PROXY' instead.
clnt = HTTPClient.new('http://myproxy:8080')
See get_content.
Get content of specified URL. It returns a String of whole result.
puts clnt.get_content('http://dev.ctor.org/')
Get content as chunks of String. It yields chunks of String.
clnt.get_content('http://dev.ctor.org/') do |chunk| puts chunk end
See head, get, post, put, delete, options, propfind, proppatch and trace. It returns a HTTP::Message instance as a response.
Do HEAD request.
res = clnt.head(uri) p res.header['Last-Modified'][0]
Do GET request with query.
query = { 'keyword' => 'ruby', 'lang' => 'en' } res = clnt.get(uri, query) p res.status p res.contenttype p res.header['X-Custom'] puts res.content
See post.
Do POST a form data.
body = { 'keyword' => 'ruby', 'lang' => 'en' } res = clnt.post(uri, body)
Do multipart file upload with POST. No need to set extra header by yourself from httpclient/2.1.4.
File.open('/tmp/post_data') do |file| body = { 'upload' => file, 'user' => 'nahi' } res = clnt.post(uri, body) end
Ruby needs to be compiled with OpenSSL.
Get content of specified URL via SSL. Just pass an URL which starts with 'https://'.
https_url = 'https://www.rsa.com' clnt.get_content(https_url)
Getting peer certificate from response.
res = clnt.get(https_url) p res.peer_cert #=> returns OpenSSL::X509::Certificate
Configuring OpenSSL options. See HTTPClient::SSLConfig for more details.
user_cert_file = 'cert.pem' user_key_file = 'privkey.pem' clnt.ssl_config.set_client_cert_file(user_cert_file, user_key_file) clnt.get_content(https_url)
Using volatile Cookies. Nothing to do. HTTPClient handles Cookies.
clnt = HTTPClient.new clnt.get_content(url1) # receives Cookies. clnt.get_content(url2) # sends Cookies if needed.
Saving non volatile Cookies to a specified file. Need to set a file at first and invoke save method at last.
clnt = HTTPClient.new clnt.set_cookie_store('/home/nahi/cookie.dat') clnt.get_content(url) ... clnt.save_cookie_store
Disabling Cookies.
clnt = HTTPClient.new clnt.cookie_manager = nil
Authentication with Web server. Supports BasicAuth, DigestAuth, and Negotiate/NTLM (requires ruby/ntlm module).
clnt = HTTPClient.new domain = 'http://dev.ctor.org/http-access2/' user = 'user' password = 'user' clnt.set_auth(domain, user, password) p clnt.get_content('http://dev.ctor.org/http-access2/login').status
Authentication with Proxy server. Supports BasicAuth and NTLM (requires win32/sspi)
clnt = HTTPClient.new(proxy) user = 'proxy' password = 'proxy' clnt.set_proxy_auth(user, password) p clnt.get_content(url)
Pass a Hash or an Array for extheader argument.
extheader = { 'Accept' => '*/*' } clnt.get_content(uri, query, extheader) extheader = [['Accept', 'image/jpeg'], ['Accept', 'image/png']] clnt.get_content(uri, query, extheader)
See head_async, get_async, post_async, put_async, delete_async, options_async, propfind_async, proppatch_async, and trace_async. It immediately returns a HTTPClient::Connection instance as a returning value.
connection = clnt.post_async(url, body) print 'posting.' while true break if connection.finished? print '.' sleep 1 end puts '.' res = connection.pop p res.status p res.content.read # res.content is an IO for the res of async method.
You can invoke get_content, get, etc. without creating HTTPClient instance.
ruby -rhttpclient -e 'puts HTTPClient.get_content(ARGV.shift)' http://dev.ctor.org/ ruby -rhttpclient -e 'p HTTPClient.head(ARGV.shift).header["last-modified"]' http://dev.ctor.org/
HTTPClient - HTTP client library. Copyright (C) 2000-2009 NAKAMURA, Hiroshi <nahi@ruby-lang.org>.
This program is copyrighted free software by NAKAMURA, Hiroshi. You can redistribute it and/or modify it under the same terms of Ruby's license; either the dual license version in 2003, or any later version.
HTTPClient - HTTP client library. Copyright (C) 2000-2009 NAKAMURA, Hiroshi <nahi@ruby-lang.org>.
This program is copyrighted free software by NAKAMURA, Hiroshi. You can redistribute it and/or modify it under the same terms of Ruby's license; either the dual license version in 2003, or any later version.
Default extheader for PROPFIND request.
How many times get_content and post_content follows HTTP redirect. 10 by default.
An array of request filter which can trap HTTP request/response. See HTTPClient::WWWAuth to see how to use it.
Creates a HTTPClient instance which manages sessions, cookies, etc.
HTTPClient.new takes 3 optional arguments for proxy url string, User-Agent String and From header String. User-Agent and From are embedded in HTTP request Header if given. No User-Agent and From header added without setting it explicitly.
proxy = 'http://myproxy:8080' agent_name = 'MyAgent/0.1' from = 'from@example.com' HTTPClient.new(proxy, agent_name, from)
You can use a keyword argument style Hash. Keys are :proxy, :agent_name and :from.
HTTPClient.new(:agent_name = 'MyAgent/0.1')
# File lib/httpclient.rb, line 340 def initialize(*args) proxy, agent_name, from = keyword_argument(args, :proxy, :agent_name, :from) @proxy = nil # assigned later. @no_proxy = nil @www_auth = WWWAuth.new @proxy_auth = ProxyAuth.new @request_filter = [@proxy_auth, @www_auth] @debug_dev = nil @redirect_uri_callback = method(:default_redirect_uri_callback) @test_loopback_response = [] @session_manager = SessionManager.new(self) @session_manager.agent_name = agent_name @session_manager.from = from @session_manager.ssl_config = @ssl_config = SSLConfig.new(self) @cookie_manager = WebAgent::CookieManager.new @follow_redirect_count = 10 load_environment self.proxy = proxy if proxy end
Returns debug device if exists. See debug_dev=.
# File lib/httpclient.rb, line 361 def debug_dev @debug_dev end
Sets debug device. Once debug device is set, all HTTP requests and responses are dumped to given device. dev must respond to << for dump.
Calling this method resets all existing sessions.
# File lib/httpclient.rb, line 369 def debug_dev=(dev) @debug_dev = dev reset_all @session_manager.debug_dev = dev end
A default method for redirect uri callback. This method is used by HTTPClient instance by default. This callback allows relative redirect such as
Location: ../foo/
in HTTP header.
# File lib/httpclient.rb, line 570 def default_redirect_uri_callback(uri, res) newuri = URI.parse(res.header['location'][0]) if https?(uri) && !https?(newuri) raise BadResponseError.new("redirecting to non-https resource") end unless newuri.is_a?(URI::HTTP) newuri = uri + newuri STDERR.puts("could be a relative URI in location header which is not recommended") STDERR.puts("'The field value consists of a single absolute URI' in HTTP spec") end puts "redirect to: #{newuri}" if $DEBUG newuri end
Sends DELETE request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 605 def delete(uri, extheader = {}, &block) request(:delete, uri, nil, nil, extheader, &block) end
Sends DELETE request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 695 def delete_async(uri, extheader = {}) request_async(:delete, uri, nil, nil, extheader) end
Sends GET request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 590 def get(uri, query = nil, extheader = {}, &block) request(:get, uri, query, nil, extheader, &block) end
Sends GET request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 677 def get_async(uri, query = nil, extheader = {}) request_async(:get, uri, query, nil, extheader) end
Retrieves a web resource.
uri |
a String or an URI object which represents an URL of web resource. |
query |
a Hash or an Array of query part of URL. e.g. { "a" => "b" } => 'host/part?a=b'. Give an array to pass multiple value like
|
extheader |
a Hash or an Array of extra headers. e.g. { 'Accept' => '/' } or [['Accept', 'image/jpeg'], ['Accept', 'image/png']]. |
&block |
Give a block to get chunked message-body of response like get_content(uri) { |chunked_body| ... }. Size of each chunk may not be the same. |
get_content follows HTTP redirect status (see HTTP::Status.redirect?) internally and try to retrieve content from redirected URL. See redirect_uri_callback= how HTTP redirection is handled.
If you need to get full HTTP response including HTTP status and headers, use get method. get returns HTTP::Message as a response and you need to follow HTTP redirect by yourself if you need.
# File lib/httpclient.rb, line 518 def get_content(uri, query = nil, extheader = {}, &block) follow_redirect(:get, uri, query, nil, extheader, &block).content end
Sends HEAD request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 585 def head(uri, query = nil, extheader = {}) request(:head, uri, query, nil, extheader) end
Sends HEAD request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 671 def head_async(uri, query = nil, extheader = {}) request_async(:head, uri, query, nil, extheader) end
Returns NO_PROXY setting String if given.
# File lib/httpclient.rb, line 411 def no_proxy @no_proxy end
Sets NO_PROXY setting String. no_proxy must be a comma separated String. Each entry must be 'host' or 'host:port' such as; HTTPClient#no_proxy = 'example.com,example.co.jp:443'
'localhost' is treated as a no_proxy site regardless of explicitly listed. HTTPClient checks given URI objects before accessing it. 'host' is tail string match. No IP-addr conversion.
You can use environment variable 'no_proxy' or 'NO_PROXY' for it.
Calling this method resets all existing sessions.
# File lib/httpclient.rb, line 426 def no_proxy=(no_proxy) @no_proxy = no_proxy reset_all end
Sends OPTIONS request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 610 def options(uri, extheader = {}, &block) request(:options, uri, nil, nil, extheader, &block) end
Sends OPTIONS request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 701 def options_async(uri, extheader = {}) request_async(:options, uri, nil, nil, extheader) end
Sends POST request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 595 def post(uri, body = '', extheader = {}, &block) request(:post, uri, nil, body, extheader, &block) end
Sends POST request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 683 def post_async(uri, body = nil, extheader = {}) request_async(:post, uri, nil, body, extheader) end
Posts a content.
uri |
a String or an URI object which represents an URL of web resource. |
body |
a Hash or an Array of body part. e.g. { "a" => "b" } => 'a=b'. Give an array to pass multiple value like
When you pass a File as a value, it will be posted as a multipart/form-data. e.g. { 'upload' => file } |
extheader |
a Hash or an Array of extra headers. e.g. { 'Accept' => '/' } or [['Accept', 'image/jpeg'], ['Accept', 'image/png']]. |
&block |
Give a block to get chunked message-body of response like post_content(uri) { |chunked_body| ... }. Size of each chunk may not be the same. |
post_content follows HTTP redirect status (see HTTP::Status.redirect?) internally and try to post the content to redirected URL. See redirect_uri_callback= how HTTP redirection is handled.
If you need to get full HTTP response including HTTP status and headers, use post method.
# File lib/httpclient.rb, line 544 def post_content(uri, body = nil, extheader = {}, &block) follow_redirect(:post, uri, nil, body, extheader, &block).content end
Sends PROPFIND request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 615 def propfind(uri, extheader = PROPFIND_DEFAULT_EXTHEADER, &block) request(:propfind, uri, nil, nil, extheader, &block) end
Sends PROPFIND request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 707 def propfind_async(uri, extheader = PROPFIND_DEFAULT_EXTHEADER) request_async(:propfind, uri, nil, nil, extheader) end
Sends PROPPATCH request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 620 def proppatch(uri, body = nil, extheader = {}, &block) request(:proppatch, uri, nil, body, extheader, &block) end
Sends PROPPATCH request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 713 def proppatch_async(uri, body = nil, extheader = {}) request_async(:proppatch, uri, nil, body, extheader) end
Returns URI object of HTTP proxy if exists.
# File lib/httpclient.rb, line 376 def proxy @proxy end
Sets HTTP proxy used for HTTP connection. Given proxy can be an URI, a String or nil. You can set user/password for proxy authentication like HTTPClient#proxy = 'user:passwd@myproxy:8080'
You can use environment variable 'http_proxy' or 'HTTP_PROXY' for it. You need to use 'cgi_http_proxy' or 'CGI_HTTP_PROXY' instead if you run HTTPClient from CGI environment from security reason. (HTTPClient checks 'REQUEST_METHOD' environment variable whether it's CGI or not)
Calling this method resets all existing sessions.
# File lib/httpclient.rb, line 390 def proxy=(proxy) if proxy.nil? @proxy = nil @proxy_auth.reset_challenge else @proxy = urify(proxy) if @proxy.scheme == nil or @proxy.scheme.downcase != 'http' or @proxy.host == nil or @proxy.port == nil raise ArgumentError.new("unsupported proxy #{proxy}") end @proxy_auth.reset_challenge if @proxy.user || @proxy.password @proxy_auth.set_auth(@proxy.user, @proxy.password) end end reset_all @session_manager.proxy = @proxy @proxy end
Sends PUT request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 600 def put(uri, body = '', extheader = {}, &block) request(:put, uri, nil, body, extheader, &block) end
Sends PUT request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 689 def put_async(uri, body = nil, extheader = {}) request_async(:put, uri, nil, body, extheader) end
Sets callback proc when HTTP redirect status is returned for get_content and post_content. default_redirect_uri_callback is used by default.
If you need strict implementation which does not allow relative URI redirection, set strict_redirect_uri_callback instead.
clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
# File lib/httpclient.rb, line 493 def redirect_uri_callback=(redirect_uri_callback) @redirect_uri_callback = redirect_uri_callback end
Sends a request to the specified URL.
method |
HTTP method to be sent. method.to_s.upcase is used. |
uri |
a String or an URI object which represents an URL of web resource. |
query |
a Hash or an Array of query part of URL. e.g. { "a" => "b" } => 'host/part?a=b' Give an array to pass multiple value like
|
body |
a Hash or an Array of body part. e.g. { "a" => "b" } => 'a=b'. Give an array to pass multiple value like
When the given method is 'POST' and the given body contains a file as a value, it will be posted as a multipart/form-data. e.g. { 'upload' => file } See HTTP::Message.file? for actual condition of 'a file'. |
extheader |
a Hash or an Array of extra headers. e.g. { 'Accept' => '/' } or [['Accept', 'image/jpeg'], ['Accept', 'image/png']]. |
&block |
Give a block to get chunked message-body of response like get(uri) { |chunked_body| ... }. Size of each chunk may not be the same. |
You can also pass a String as a body. HTTPClient just sends a String as a HTTP request message body.
When you pass an IO as a body, HTTPClient sends it as a HTTP request with chunked encoding (Transfer-Encoding: chunked in HTTP header). Bear in mind that some server application does not support chunked request. At least cgi.rb does not support it.
# File lib/httpclient.rb, line 659 def request(method, uri, query = nil, body = nil, extheader = {}, &block) uri = urify(uri) if block filtered_block = proc { |res, str| block.call(str) } end do_request(method, uri, query, body, extheader, &filtered_block) end
Sends a request in async style. request method creates new Thread for HTTP connection and returns a HTTPClient::Connection instance immediately.
Arguments definition is the same as request.
# File lib/httpclient.rb, line 727 def request_async(method, uri, query = nil, body = nil, extheader = {}) uri = urify(uri) do_request_async(method, uri, query, body, extheader) end
Resets internal session for the given URL. Keep-alive connection for the site (host-port pair) is disconnected if exists.
# File lib/httpclient.rb, line 734 def reset(uri) uri = urify(uri) @session_manager.reset(uri) end
Resets all of internal sessions. Keep-alive connections are disconnected.
# File lib/httpclient.rb, line 740 def reset_all @session_manager.reset_all end
Sets credential for Web server authentication.
domain |
a String or an URI to specify where HTTPClient should use this |
credential. If you set uri to nil, HTTPClient uses this credential wherever a server requires it.
user |
username String. |
passwd |
password String. |
You can set multiple credentials for each uri.
clnt.set_auth('http://www.example.com/foo/', 'foo_user', 'passwd') clnt.set_auth('http://www.example.com/bar/', 'bar_user', 'passwd')
Calling this method resets all existing sessions.
# File lib/httpclient.rb, line 444 def set_auth(domain, user, passwd) uri = urify(domain) @www_auth.set_auth(uri, user, passwd) reset_all end
Deprecated. Use set_auth instead.
# File lib/httpclient.rb, line 451 def set_basic_auth(domain, user, passwd) uri = urify(domain) @www_auth.basic_auth.set(uri, user, passwd) reset_all end
Sets credential for Proxy authentication.
user |
username String. |
passwd |
password String. |
Calling this method resets all existing sessions.
# File lib/httpclient.rb, line 462 def set_proxy_auth(user, passwd) @proxy_auth.set_auth(user, passwd) reset_all end
A method for redirect uri callback. How to use:
clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
This callback does not allow relative redirect such as
Location: ../foo/
in HTTP header. (raises BadResponseError instead)
# File lib/httpclient.rb, line 553 def strict_redirect_uri_callback(uri, res) newuri = URI.parse(res.header['location'][0]) if https?(uri) && !https?(newuri) raise BadResponseError.new("redirecting to non-https resource") end unless newuri.is_a?(URI::HTTP) raise BadResponseError.new("unexpected location: #{newuri}", res) end puts "redirect to: #{newuri}" if $DEBUG newuri end
Sends TRACE request to the specified URL. See request for arguments.
# File lib/httpclient.rb, line 625 def trace(uri, query = nil, body = nil, extheader = {}, &block) request('TRACE', uri, query, body, extheader, &block) end
Sends TRACE request in async style. See request_async for arguments. It immediately returns a HTTPClient::Connection instance as a result.
# File lib/httpclient.rb, line 719 def trace_async(uri, query = nil, body = nil, extheader = {}) request_async(:trace, uri, query, body, extheader) end
Generated with the Darkfish Rdoc Generator 2.