Ruby on Rails and PyTorch: Building Reliable AI Features

Written by James Britt

A Rails help desk already handles tickets, checks user access and allows agents to view their queues. Suppose the next step is a suggested category for new messages. A PyTorch model could give recommendations while Rails continues to manage the application around it.

What we now have is a good foundation for combining Ruby on Rails and PyTorch. We will need an interface to communicate between our apps, some method of handling missing predictions (i.e., cases where no prediction was made by the model), and a person responsible for running each application. This is a much smaller decision than moving an entire Rails application into Python.

In addition to detailing the model task, the hiring brief for teams planning to hire pytorch developers should outline the Rails integration. The ideal candidate should be able to describe what information your Ruby application sends, what it receives, and what happens when the prediction service is unavailable.

What Ruby on Rails and PyTorch each handle

PyTorch is a tool for building and training machine learning models. Its introductory tutorials cover for loading data, constructing models and optimizing them. For this example, the model will run in a separate Python service. Your Rails application will call its API.

Separating your Rails application and PyTorch model is a design choice for this tutorial. It enables your Rails development team to continue using its existing Rails application and to deploy the model separately. Ruby-Doc Learn’s Python comparison offers language background. The article Ruby on Rails vs Python compares the differences between a language and a framework.

Responsibility	Suggested owner
Accounts, permissions and ticket records	Rails application
Text preprocessing and model inference	Python prediction service
Whether a suggestion changes a ticket	Rails product logic
Training data and model evaluation	Machine-learning team, with product input

Establish clearly defined business choices. Even though a classifier may suggest “billing”, your application determines whether to present that suggestion, assign a queue, or require review. Each action has its own set of acceptance criteria.

Agree the prediction contract before training

Begin with a simple request with only the text needed for classification. Sending whole database records can expose information the model does not require. Establish how large the maximum input size can be and which fields must be removed prior to your Rails application releasing the data.

The response contract needs to have a label, a score and a model version. A successful result may look something like label: "billing", score: 0.84, and model_version: "ticket-v1". These values illustrate a contract; they are not measured results.

A score does not necessarily represent a percentage probability of being correct. Ask how the score is calculated and whether calibration has been evaluated. Define what an unknown label, an empty response or an unsupported model version represents to your Rails application.

Document your error contract too: invalid input, authentication errors, rate limiting, temporary service failure. Utilize fixture responses so that your Rails development team can continue developing the app while the model is still in testing.

A small Ruby client for the prediction API

Add the HTTP call to a service object, i.e. app/services/ticket_classifier.rb. This illustrative client utilizes Ruby’s Net::HTTP API. Configure your trusted HTTPS endpoint in ML_PREDICT_URL and its service token in ML_API_TOKEN.

require "net/http"
require "json"

class TicketClassifier
  class PredictionError < StandardError; end
  LABELS = %w[billing technical other].freeze

  def call(text)
    uri = URI(ENV.fetch("ML_PREDICT_URL"))
    raise ArgumentError, "HTTPS required" unless uri.is_a?(URI::HTTPS)

    request = Net::HTTP::Post.new(uri)
    request["Content-Type"] = "application/json"
    request["Accept"] = "application/json"
    request["Authorization"] = "Bearer #{ENV.fetch("ML_API_TOKEN")}"
    request.body = JSON.generate(text: text)

    http = Net::HTTP.new(uri.hostname, uri.port)
    http.use_ssl = true
    http.open_timeout = 1
    http.read_timeout = 2
    http.write_timeout = 2
    http.max_retries = 0

    response = http.start { |session| session.request(request) }
    unless response.code == "200"
      raise PredictionError, "Inference HTTP #{response.code}"
    end

    result = JSON.parse(response.body)
    valid = result.is_a?(Hash) &&
      LABELS.include?(result["label"]) &&
      result["model_version"].is_a?(String) &&
      !result["model_version"].empty? &&
      result["score"].is_a?(Numeric) &&
      result["score"].finite? &&
      result["score"].between?(0, 1)
    raise PredictionError, "Invalid prediction" unless valid

    result
  rescue JSON::ParserError
    raise PredictionError, "Invalid JSON"
  end
end

Once you have applied input limitations and data filtering, call TicketClassifier.new.call(ticket.body) from the part of your application responsible for classification. The example validates the agreed upon fields and raises exceptions on unexpected outcomes. These example timeout values are intended to be adjusted after measuring. Individual socket timeouts do not impose a total request deadline.

Prior to production, implement response-size limits, a total deadline for each request, request tracing and the behavior you want your product to exhibit upon failure. Ensure the endpoint remains within trusted configurations. Redact any credentials or ticket body from log entries. Network exceptions should always represent a failure rather than become successful classifications.

Decide whether the Rails request should wait

Showing a suggestion after a ticket is created often belongs in a background job. Agents can read tickets while classification runs. Display “suggestion pending” or leave the field blank until a valid response arrives.

Active Job allows Rails to abstract away queuing behavior. Select your desired queue adapter and workers for your chosen deployment strategy and establish intentional retry behaviors. Queues do not inherently guarantee reliability of an external API.

Utilize bounded retries for temporary failures where repeating the operation is safe. Avoid indefinitely attempting to retry bad input. If the service tracks requests, charges per call or produces another side effect, decide how duplicates will be recognized.

If a ticket changes during classification, only save the suggested category when the current revision matches the original revision sent with the ticket. Otherwise a response can associate a stale category with newly edited content.

Synchronous calls may suit interactions that require an immediate answer. Assess the overall impact on Rails response time including the slowest requests. Decide whether a timeout should present a fallback or pause the action.

What to ask a PyTorch developer

Have candidates discuss this ticket feature. Provide projected traffic, a representative dataset sample and examples of costly mistakes. Having billing messages sent to the wrong queue provides a more useful conversation than “which architecture is best?”

Assess evidence in four areas:

• Baseline judgment: are rules or simpler classifiers enough to address the problem prior to using neural models?
• Evaluation: how will they prevent related tickets from appearing both in the test and train sets? Also evaluate mistakes across meaningful categories.
• Serving: can they evaluate response times and memory usage on the hardware you plan to operate?
• Integration: can they document preprocessing? Version their response contracts? Help your Ruby team troubleshoot failed requests?

Time-bounded exercises can use mock prediction endpoints and several difficult responses. Have candidates explain malformed JSON, unrecognized categories, timeouts and results returned by older versions of the model. Establish agreement on exercise scope and assessment criteria prior to them beginning.

Measure the feature inside the Rails workflow

Measuring model performance outside of your Rails workflow is just one aspect of meeting your acceptance criteria. Track how often agents accept or correct suggestions. Track how long results take to return. Track how frequently the service experiences failure. Establish thresholds based upon your workflow instead of copying latency goals from other products.

Track the model version alongside each suggestion so future updates can be tracked down. Separate human corrections from predictions. Otherwise labels generated by earlier models will quietly become accepted truth for subsequent training runs.

Displaying suggestions for agents to approve provides feedback for your development team. Automatic assignment can follow later if supporting data exists. Maintain separate metrics for tickets where incorrect categories would cause significantly greater delays.

Choose a current deployment route

Do not make TorchScript your default simply because an older guide recommends it. As noted in the official PyTorch release notes, TorchScript is deprecated and points users toward torch.export for replacing its tracing and scripting APIs.

Not all Rails integrations need model export. A Python service can run models and present a documented HTTP endpoint. Export and compilation decisions depend on your serving environment and model. Request demonstrated benefits and compatibility checks prior to adding them.

You can find reference material about supported behavior and dependencies within the PyTorch source repository.

Record actual framework version used, model artifact created and preprocessing code used in production.

Keep ownership clear when working with a partner

Your Rails team owns application behavior and its corresponding acceptance criteria. The model developer should document training recipes, evaluation data, preprocessing and service operation. Identify who monitors failures and who has authority to roll back each deployment.

Include integration code, fixtures and operating instructions in repositories controlled by your team. Request documentation another engineer can follow: Start the service, send a sample request, interpret the result and restore the previous model.

Develop one narrow feature at a time. Run it in staging, test failure paths then release it to a limited group. A useful first milestone is a Rails workflow that remains understandable while predictions arrive late or fail. Model complexity can follow.