Written by James Britt
Let’s say you’ve got an inbox for support inside your Rails app. A customer has sent 6 messages. Two people have responded. Now someone else opens up that ticket and wants to know what happened. A little bit of context would really help.
If you want to get that functionality into your app (without switching languages), send the whole relevant thread off to a language model. The model returns a draft summary for the next agent to review. Your Rails app manages all the records, permissions and interfaces that happen within that transaction.
Whether that feature helps your team is a more useful question than whether Ruby will replace Python. Your app already exists. You and your team know how to keep it going. Now you’re just trying to decide if some new AI feature will actually improve things.
Beyond Model Training – Ruby Has a Role
Model training and using a model in a product are two completely different types of work. A Rails developer putting together a summary button needs to decide which messages to include, when to send the request and where to display the results.
None of these decisions go away simply because the model gets smarter. A good summary is still a problem if it includes conversations that the user shouldn’t ever have had access to.
For a team currently using Ruby, there is a practical reason to keep this work in the application they understand. They can reuse its permission checks and follow its conventions for jobs, records and errors. A separate model service can sit behind the application if needed.
Take Another Look at Your Toolkit
Ruby has many options for connecting with AI, so names of gems can confuse you.
For instance, OpenAI’s official Ruby SDK is available through the openai gem. The ruby-openai gem is a completely different project. Anthropic has released an official Ruby SDK through the anthropic gem. Before copying an example, figure out which gem was referenced in it. Similar gem names do not mean identical APIs.
For machine learning outside of hosted language models, Rumale provides classification, regression, clustering and data pre-processing tools. Rumale relies on numerical array dependencies, so describing the whole stack as pure Ruby misses an important detail. Projects working with labeled datasets might find it worth looking into.
Torch.rb provides a Ruby interface powered by LibTorch for deep learning work. You’ll need to install native libraries and ensure that compatible versions exist for them. Check those requirements in your deployment environment before committing to the library.
Hosted API clients and local machine-learning libraries involve different commitments. One relies on a vendor’s service. The other places responsibility for running the work onto you. The correct choice for your specific project will ultimately come down to what you’re building and who will be responsible for maintaining it.
Leave Room for Mistakes
In the case of our support inbox, perhaps the first few successful demonstrations look great. The summaries are concise, easily readable, and appear in the right locations.
However, then test one where the customer changed their mind mid-conversation. Test one where an agent discusses possible refunds but doesn’t promise anything. Do those differences remain evident in the summaries?
Those scenarios are worthy of inclusion in your assessment processes. Additionally, test missing replies, ambiguous references, and attempts by customers to issue commands to the model.
The Rails implementation needs its own checks. A background job might fail and retry, or finish after another reply has arrived from the customer. Linking the summary to the specific conversation version that was used allows an agent to quickly determine if it remains valid.
For an initial release, provide a draft to the agent for editing or discarding. Keep the original messages accessible. The corrections will help you identify where the feature needs work.
Human Review Belongs Inside Your Product
Publishing tools represent another example. Ruby can organise drafts, source notes and editorial approvals while a model suggests tags or descriptions.
Editors may also use an AI checker during review. However, editors should exercise caution as a detector score cannot guarantee authorship nor verify statements contained within articles. Research has demonstrated false positives and instances where generated content evaded detection.
Your application should allow editors to review evidence supporting each segment of content presented by a model. Saved source notes and revision histories can help editors answer questions that detector scores cannot.
Readability also applies when coding. Your fellow maintainer should be able to locate the rules governing which information is provided to the model as well as how its response is processed. A Rails team can write those rules in familiar Ruby code. It does not provide editorial judgment by default.
Begin with One Useful Task
Most teams adding an assistant to a Rails product do not need to train a trillion-parameter model. They need to find out whether a smaller feature helps someone do their job.
For your summary button, measure the time saved after checking and correcting the output. If agents consistently rewrite your model’s output, then rapid responses from your API are less likely to aid your users.
Ruby can remain useful in AI projects for the reasons a team chose it in the first place: familiar tools, readable code and a way to build features they can maintain. There is plenty of application work around a model, wherever that model runs.