{"id":1799,"date":"2026-11-09T09:48:43","date_gmt":"2026-11-09T09:48:43","guid":{"rendered":"https:\/\/ruby-doc.org\/learn\/?p=1799"},"modified":"2026-11-09T09:49:01","modified_gmt":"2026-11-09T09:49:01","slug":"concurrent-http-requests-in-ruby","status":"publish","type":"post","link":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/","title":{"rendered":"Concurrent HTTP Requests in Ruby Using Async, Threads and Proxies"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-1024x576.jpeg\" alt=\"\" class=\"wp-image-1800\" srcset=\"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-1024x576.jpeg 1024w, https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-300x169.jpeg 300w, https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-768x432.jpeg 768w, https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-1536x864.jpeg 1536w, https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b-2048x1152.jpeg 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n<p>Suppose your Ruby program is checking 5,000 URLs. If it uses <code>Net::HTTP<\/code> to loop over this list of URLs, a slow server could block subsequent URL checks even if the next URL belongs to a different host.<\/p>\n<p>Running concurrent HTTP requests in Ruby allows your program to start looking at the next URL while the current one is blocked waiting for a socket. A fixed-size thread pool is a good starting point. Fiber-based programming with Async may be worthwhile if you&#8217;ve got a fiber scheduler running elsewhere in your application or you&#8217;re maintaining many active connections.<\/p>\n<p>Look at how often you create new connections vs. how many times you reuse existing ones. A fresh HTTPS connection can involve a DNS lookup, a TCP handshake and a TLS handshake. If multiple of your URLs are being fetched from the same host, you can save yourself the cost of these handshakes by reusing a session for that host. Compare how long the batch takes and how many requests succeed.<\/p>\n<h2>Overlapping HTTP requests with Ruby threads<\/h2>\n<p>In a CRuby Ractor, the global VM lock (GVL) allows only one thread to execute Ruby code at a time. However, since one thread can release the GVL while waiting on ordinary socket I\/O, other workers can process their respective requests concurrently. If your profiling tool indicates that the majority of the execution time is spent parsing very large responses, increasing the number of fetching threads can increase the number of responses left to parse. Measure the fetch and parse phases independently before attempting to move CPU-bound work to other processes or Ractors.<\/p>\n<p>Have each worker in the pool use a shared <a href=\"https:\/\/docs.ruby-lang.org\/en\/4.0\/Thread\/Queue.html\"><code>Thread::Queue<\/code><\/a>. <code>close<\/code> signals to all workers that there will be no more jobs sent. The workers continue to drain the queue until it&#8217;s empty and <code>pop<\/code> returns <code>nil<\/code>. The queue holds URL strings so <code>nil<\/code> can signal shutdown.<\/p>\n<p>Here is an example status checker that reads URLs from the command line and runs four workers. Put this example in a file called <code>check_status.rb<\/code> and run <code>ruby check_status.rb<\/code> with a few URLs you control that support <code>HEAD<\/code>. Each job opens its own connection. This example uses APIs available in CRuby 3.x and 4.0.<\/p>\n<pre class=\"wp-block-code\"><code class=\"language-ruby\">require &quot;net\/http&quot;\nrequire &quot;openssl&quot;\nrequire &quot;thread&quot;\n\njobs = Thread::Queue.new\nARGV.each { |url| jobs &lt;&lt; url }\njobs.close\nresults = Thread::Queue.new\n\nworkers = Array.new(4) do\n  Thread.new do\n    while (url = jobs.pop)\n      begin\n        uri = URI(url)\n        raise ArgumentError unless uri.is_a?(URI::HTTP) &amp;&amp; uri.hostname\n\n        http = Net::HTTP.new(uri.hostname, uri.port, nil)\n        http.use_ssl = uri.scheme == &quot;https&quot;\n        http.open_timeout = 3\n        http.read_timeout = 5\n        http.write_timeout = 5\n        http.max_retries = 0\n        results &lt;&lt; [url, http.head(uri.request_uri).code]\n      rescue URI::Error, ArgumentError, Timeout::Error, SocketError,\n             SystemCallError, IOError, Net::ProtocolError, Net::HTTPBadResponse,\n             Net::HTTPHeaderSyntaxError, OpenSSL::SSL::SSLError =&gt; error\n        results &lt;&lt; [url, error.class.name]\n      end\n    end\n  end\nend\n\nworkers.each(&amp;:value)\nresults.close\nwhile (result = results.pop)\n  p result\nend<\/code><\/pre>\n<p>The checker prints results in completion order. The rescue is within the loop, so a timeout on one job will allow the worker to immediately process the next job. If you transition from an argument list of small size to large files, start the workers prior to providing a <code>Thread::SizedQueue<\/code>. The producer waits when the queue fills.<\/p>\n<p>Drain or write results as soon as possible since otherwise you risk having an unbounded queue grow.<\/p>\n<p>To test how your application performs against a single host, consider assigning each worker its own <code>Net::HTTP.start<\/code> session and reusing it for all path names served by that domain. There is a danger of sharing a single <code>Net::HTTP<\/code> object among multiple workers interfering with how they service the same connection. If you already use concurrent-ruby, a <code>Concurrent::FixedThreadPool<\/code> with a specified <code>max_queue<\/code> and <code>fallback_policy<\/code> enables you to similarly limit pending work.<\/p>\n<h2>When to use Async fibers<\/h2>\n<p>Ruby 3.0 introduced the fiber scheduler interface. Non-blocking fibers can yield at supported operations like waiting on a socket or sleeping. The scheduler then runs another fiber on the same thread and resumes the waiting fiber when it can continue.<\/p>\n<p>Your installed gem provides the scheduler, but check its minimum Ruby version before choosing a release.<\/p>\n<p>The Async project&#8217;s <a href=\"https:\/\/socketry.github.io\/async\/reference\/Async\/Semaphore\/index.html\"><code>Async::Semaphore<\/code> example with <code>Net::HTTP<\/code><\/a> demonstrates how to restrict the number of tasks using sockets simultaneously. When grouping tasks using <code>Async::Barrier<\/code>, call <code>wait<\/code> to collect tasks and insert <code>stop<\/code> in an <code>ensure<\/code> block so any remaining tasks are terminated when exiting early.<\/p>\n<p>Verify any adapters calling blocking native code. If an adapter calls blocking native code, it will block the scheduler&#8217;s thread. All fibers running on that thread will wait as well. Verify task state as well. True thread variables are shared by all fibers running on that thread while <code>Thread.current[:key]<\/code> is unique per fiber in Ruby.<\/p>\n<table>\n<thead>\n<tr class=\"header\">\n<th>What you currently have<\/th>\n<th>Next step<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr class=\"odd\">\n<td>A stand-alone link checker or health probe<\/td>\n<td>A small, fixed-size thread pool<\/td>\n<\/tr>\n<tr class=\"even\">\n<td>Many open connections or an application that is already using a fiber scheduler<\/td>\n<td>Async with a semaphore<\/td>\n<\/tr>\n<tr class=\"odd\">\n<td>A codebase that is already using Typhoeus<\/td>\n<td>Use <code>Typhoeus::Hydra<\/code> to manage concurrency of libcurl<\/td>\n<\/tr>\n<tr class=\"even\">\n<td>Fast downloads followed by slow parsing<\/td>\n<td>Separate processing phase<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>Add timeouts and throttle each host<\/h2>\n<p>A concurrency limit restricts how many requests can occur at once. A rate limit determines how frequently a request can initiate. A fast server can still receive many requests per second from a pool of four workers. Use a global worker limit and a smaller per-host limit when a batch spans several destinations. If a server supplies a quota, use a shared limiter for all workers under that quota. An API may count usage based on token, account or IP address. Track usage based on what the server tracks.<\/p>\n<p>Provide <code>Net::HTTP<\/code> with explicit open, read and write timeouts. The default value for <code>open_timeout<\/code> is 60 seconds. The read timeout applies to each individual blocking read, so a server can keep a request alive longer than expected if it sends data very slowly. To help identify if a request meets its deadline, keep track of elapsed time separately. The <a href=\"https:\/\/ruby-doc.org\/learn\/build-a-reliable-ruby-http-fetcher-with-nethttp\/\">Ruby Doc Guide for Building a Reliable <code>Net::HTTP<\/code> Fetcher<\/a> explains how each setting affects fetchers in practice.<\/p>\n<p>Track what occurs if you receive a 429 after issuing requests. If the response contains a usable <code>Retry-After<\/code> header, suspend new requests under the same quota until the requested duration has passed. The header may include either an integer representing seconds or an HTTP date. Without a usable <code>Retry-After<\/code>, use capped exponential back-off with jitter. Cap retry attempts and store them in one location. As indicated earlier, this example sets <code>max_retries = 0<\/code> so <code>Net::HTTP<\/code> won&#8217;t repeat an idempotent request after certain transport errors while your job runner continues to retry.<\/p>\n<h2>Datacenter proxies and connection reuse<\/h2>\n<p>Use proxies if routing matters. For example, your QA team needs to check the German version of a page from Germany or an uptime check needs independent network locations to view web pages from. Datacenter IPs are assigned to hosting networks, which are treated differently than residential traffic by some destinations. Validate that the destination permits desired access and validate that you maintain identical rate limits throughout the job regardless of additional routed paths.<\/p>\n<p>You can provide proxy host, port, username and password to <code>Net::HTTP.new<\/code> after specifying destination host and port. This example passes <code>nil<\/code> for proxy since it wants to establish direct connections. Leaving the proxy argument at its default allows <code>Net::HTTP<\/code> to use the proxy configuration from the environment.<\/p>\n<p>Keep proxy credentials in configuration and out of the logs.<\/p>\n<p>When connecting through an HTTP proxy, HTTPS clients typically open a <code>CONNECT<\/code> tunnel and run TLS through it. If the gateway selects an exit IP when a tunnel is established, subsequent requests on that persistent connection will use the same exit IP. Plain HTTP gateways may be capable of selecting a route for each request. Verify the behaviour with an endpoint you control before assuming your gateway behaves as expected.<\/p>\n<p>ProxyEmpire&#8217;s pool of <a href=\"https:\/\/proxyempire.io\/rotating-datacenter-proxies\/\">rotating datacenter IPs<\/a>, for example, offers a new IP on every request or sticky sessions, with country-level targeting. Sticky sessions may keep an authorised paginated run on the same route. For independent location checks, a fresh connection lets the gateway choose another exit, with an additional handshake.<\/p>\n<p>That selected exit route may repeat depending on the provider.<\/p>\n<h2>Test a small run before scaling it up<\/h2>\n<p>Test with a small batch the destination permits. Determine throughput, latency, and failure rates per host for various pool sizes. Count 429s and challenge pages as failures, even when a challenge page returns a 200 status. Verify the behaviour with an empty input queue and a timed-out request. Check what happens when you cancel the job halfway through. Record status, elapsed time and attempt counts. Exclude tokens and cookies from output.<\/p>\n<p>Cache responses for repeat checks and send <code>If-None-Match<\/code> or <code>If-Modified-Since<\/code> when the server supplies validators. If it replies with <code>304 Not Modified<\/code>, reuse the cached response.<\/p>\n<p>Check the site&#8217;s terms and <code>robots.txt<\/code> before automating requests. Use an official API or feed where required.<\/p>","protected":false},"excerpt":{"rendered":"<p>Use a fixed thread pool or Async fibers to overlap Ruby HTTP requests. Compare per-host pacing, timeouts and connection reuse when a proxy is involved.<\/p>\n","protected":false},"author":3,"featured_media":1800,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1799","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ruby-tips"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":7}},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Concurrent HTTP Requests in Ruby with Threads and Async<\/title>\n<meta name=\"description\" content=\"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Concurrent HTTP Requests in Ruby with Threads and Async\" \/>\n<meta property=\"og:description\" content=\"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/\" \/>\n<meta property=\"og:site_name\" content=\"Ruby-Doc Learn\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-09T09:48:43+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-11-09T09:49:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"2400\" \/>\n\t<meta property=\"og:image:height\" content=\"1350\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"James Britt\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"James Britt\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/\"},\"author\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/james-britt\\\/#james-britt\"},\"headline\":\"Concurrent HTTP Requests in Ruby Using Async, Threads and Proxies\",\"datePublished\":\"2026-11-09T09:48:43+00:00\",\"dateModified\":\"2026-11-09T09:49:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/\"},\"wordCount\":1349,\"publisher\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg\",\"articleSection\":[\"Ruby tips\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/\",\"name\":\"Concurrent HTTP Requests in Ruby with Threads and Async\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg\",\"datePublished\":\"2026-11-09T09:48:43+00:00\",\"dateModified\":\"2026-11-09T09:49:01+00:00\",\"description\":\"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg\",\"contentUrl\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg\",\"width\":2400,\"height\":1350},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/concurrent-http-requests-in-ruby\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Concurrent HTTP Requests in Ruby Using Async, Threads and Proxies\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#website\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/\",\"name\":\"Ruby-Doc Learn\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#organization\",\"name\":\"Ruby-Doc Learn\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/ruby-doc-logo-transparent.png\",\"contentUrl\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/ruby-doc-logo-transparent.png\",\"width\":1650,\"height\":305,\"caption\":\"Ruby-Doc Learn\"},\"image\":{\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/james-britt\\\/#james-britt\",\"name\":\"James Britt\",\"image\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/jgb_self-portrait-20140914.png\",\"description\":\"Creator and long-term maintainer of Ruby-Doc.org, founder of Neurogami, and author of practical Ruby tutorials for Ruby-Doc Learn.\",\"sameAs\":[\"https:\\\/\\\/jamesbritt.com\\\/\",\"https:\\\/\\\/neurogami.com\\\/\",\"https:\\\/\\\/www.oreilly.com\\\/pub\\\/au\\\/2595\",\"https:\\\/\\\/www.rubyevents.org\\\/profiles\\\/james-britt\"],\"url\":\"https:\\\/\\\/ruby-doc.org\\\/learn\\\/james-britt\\\/\",\"jobTitle\":\"Ruby developer and writer\",\"knowsAbout\":[\"Ruby programming language\",\"Ruby documentation\",\"JRuby\",\"Open Sound Control\",\"Software development\"],\"affiliation\":{\"@type\":\"Organization\",\"name\":\"Ruby-Doc.org\",\"url\":\"https:\\\/\\\/ruby-doc.org\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Concurrent HTTP Requests in Ruby with Threads and Async","description":"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/","og_locale":"en_US","og_type":"article","og_title":"Concurrent HTTP Requests in Ruby with Threads and Async","og_description":"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.","og_url":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/","og_site_name":"Ruby-Doc Learn","article_published_time":"2026-11-09T09:48:43+00:00","article_modified_time":"2026-11-09T09:49:01+00:00","og_image":[{"width":2400,"height":1350,"url":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg","type":"image\/jpeg"}],"author":"James Britt","twitter_card":"summary_large_image","twitter_misc":{"Written by":"James Britt","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#article","isPartOf":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/"},"author":{"@id":"https:\/\/ruby-doc.org\/learn\/james-britt\/#james-britt"},"headline":"Concurrent HTTP Requests in Ruby Using Async, Threads and Proxies","datePublished":"2026-11-09T09:48:43+00:00","dateModified":"2026-11-09T09:49:01+00:00","mainEntityOfPage":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/"},"wordCount":1349,"publisher":{"@id":"https:\/\/ruby-doc.org\/learn\/#organization"},"image":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#primaryimage"},"thumbnailUrl":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg","articleSection":["Ruby tips"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/","url":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/","name":"Concurrent HTTP Requests in Ruby with Threads and Async","isPartOf":{"@id":"https:\/\/ruby-doc.org\/learn\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#primaryimage"},"image":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#primaryimage"},"thumbnailUrl":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg","datePublished":"2026-11-09T09:48:43+00:00","dateModified":"2026-11-09T09:49:01+00:00","description":"Run concurrent HTTP requests in Ruby with a thread pool or Async fibers. Set timeouts, respect per-host limits and understand how proxies reuse connections.","breadcrumb":{"@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#primaryimage","url":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg","contentUrl":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/11\/real_a1208e3e-9dfd-4b5e-aa20-68c7e1269a7b.jpeg","width":2400,"height":1350},{"@type":"BreadcrumbList","@id":"https:\/\/ruby-doc.org\/learn\/concurrent-http-requests-in-ruby\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ruby-doc.org\/learn\/"},{"@type":"ListItem","position":2,"name":"Concurrent HTTP Requests in Ruby Using Async, Threads and Proxies"}]},{"@type":"WebSite","@id":"https:\/\/ruby-doc.org\/learn\/#website","url":"https:\/\/ruby-doc.org\/learn\/","name":"Ruby-Doc Learn","description":"","publisher":{"@id":"https:\/\/ruby-doc.org\/learn\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ruby-doc.org\/learn\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ruby-doc.org\/learn\/#organization","name":"Ruby-Doc Learn","url":"https:\/\/ruby-doc.org\/learn\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ruby-doc.org\/learn\/#\/schema\/logo\/image\/","url":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/09\/ruby-doc-logo-transparent.png","contentUrl":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/09\/ruby-doc-logo-transparent.png","width":1650,"height":305,"caption":"Ruby-Doc Learn"},"image":{"@id":"https:\/\/ruby-doc.org\/learn\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ruby-doc.org\/learn\/james-britt\/#james-britt","name":"James Britt","image":"https:\/\/ruby-doc.org\/learn\/wp-content\/uploads\/2026\/09\/jgb_self-portrait-20140914.png","description":"Creator and long-term maintainer of Ruby-Doc.org, founder of Neurogami, and author of practical Ruby tutorials for Ruby-Doc Learn.","sameAs":["https:\/\/jamesbritt.com\/","https:\/\/neurogami.com\/","https:\/\/www.oreilly.com\/pub\/au\/2595","https:\/\/www.rubyevents.org\/profiles\/james-britt"],"url":"https:\/\/ruby-doc.org\/learn\/james-britt\/","jobTitle":"Ruby developer and writer","knowsAbout":["Ruby programming language","Ruby documentation","JRuby","Open Sound Control","Software development"],"affiliation":{"@type":"Organization","name":"Ruby-Doc.org","url":"https:\/\/ruby-doc.org\/"}}]}},"_links":{"self":[{"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/posts\/1799","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/comments?post=1799"}],"version-history":[{"count":1,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/posts\/1799\/revisions"}],"predecessor-version":[{"id":1801,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/posts\/1799\/revisions\/1801"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/media\/1800"}],"wp:attachment":[{"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/media?parent=1799"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/categories?post=1799"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ruby-doc.org\/learn\/wp-json\/wp\/v2\/tags?post=1799"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}