Ruby to PyTorch: test the process boundary before CUDA

A Ruby application does not have to become a Python application to use a numerical library. It can send one well-defined request to a Python worker and read the result. The interesting part is deciding what crosses that boundary—and what happens when the worker refuses it.

This example adds two vectors using real PyTorch tensors. Ruby starts the worker, sends JSON through standard input and checks its exit status. The CPU path was executed locally. The same worker has an explicit CUDA option, but no NVIDIA device was available for a GPU run. There is no speedup claim hidden in the vector addition.

It is a small place to establish an integration contract before dealing with device memory, batching or a long-lived GPU process.

Send data, not a generated command

The Ruby side uses Open3.capture3. The executable, script path and device are separate arguments. The vectors travel over stdin as JSON:

output, error, status = Open3.capture3(
  python, worker, '--device', device,
  stdin_data: JSON.generate({a: a, b: b})
)
raise "worker failed: #{error.strip}" unless status.success?

There is no shell expression to interpolate an input vector into. Keep python and the worker path under application control; separating arguments does not make an arbitrary executable safe to run. The device option is also checked by Python's argument parser.

Reserve stdout for the response. Diagnostic text belongs on stderr. Otherwise a harmless progress message can turn a valid JSON reply into a parse error. A nonzero exit status becomes an exception on the Ruby side before any output is interpreted as a successful result.

The client then checks the response structure, reported device and dtype. It is a teaching boundary, not a complete hostile-process sandbox: it assumes the local worker file is trusted and does not impose a deadline on execution.

Make numeric assumptions part of the protocol

The Python worker accepts exactly two fields, a and b. They must contain equally sized, nonempty vectors, with no more than 4,096 elements. Elements must be finite numbers between negative and positive 1e20. Booleans are rejected even though Python treats bool as an integer subtype.

These limits keep the local example bounded. They are not performance tuning. The worker also rejects input larger than 256 KiB before parsing it. A Ruby caller can still allocate a large JSON string before sending it; callers need their own limits if requests come from outside the application.

The operation itself is short:

with torch.inference_mode():
    left = torch.tensor(a, dtype=torch.float32, device=device)
    right = torch.tensor(b, dtype=torch.float32, device=device)
    result = (left + right).cpu().tolist()

The explicit dtype matters. JSON numbers arrive as Python values, but conversion to float32 changes their representation. The result for 0.1 + 0.2 should be compared within a tolerance, not against an exact decimal string. The tests include that case. For accounting values requiring exact decimal arithmetic, this protocol would be the wrong representation.

Moving the result back to CPU is part of completing the request. It also means this design cannot keep an intermediate tensor on the GPU for the next Ruby call. That is an architectural cost, even before measuring serialization or process startup.

Run the CPU integration first

Save client.rb, worker.py, test_client.rb and test_worker.py in one directory. The example notes record the tested environment and dependency versions: Ruby 3.3.2, Minitest 6.0.6, Python 3.11.5 and PyTorch 2.9.1.

Run ruby client.rb with PYTHON pointing to the Python interpreter containing PyTorch. The observed response was:

{"device":"cpu","dtype":"float32","values":[4.0,0.0,0.75]}

ruby test_client.rb --seed 42 passed six tests with 13 assertions, including actual subprocess calls for successful arithmetic, rounding, wrong lengths, nonnumeric input, empty vectors and an invalid device argument. python3 -B -m unittest -v test_worker.py passed three additional tests for input limits, schema checks and refusal to silently replace unavailable CUDA with CPU.

Starting and importing PyTorch for every request is intentionally expensive. This version makes each call easy to inspect and gives it a fresh process. It is a reference integration, not a throughput-oriented worker pool. The test duration includes those starts and should not be reported as tensor compute time.

Request CUDA explicitly

The accepted device names are cpu and cuda:0. When CUDA is requested, the worker checks availability and fails if it is absent. It does not quietly run on CPU and return a response that looks like a GPU measurement.

On a machine with a compatible NVIDIA setup, the same vector_add method accepts device: 'cuda:0'. That path was not executed here. Follow PyTorch's installation instructions for the target platform and then rerun the integration tests with an added GPU oracle before relying on it.

PyTorch's CUDA semantics explain device placement, transfers and asynchronous execution. For this protocol, explicit placement plus the copy back to CPU makes the ownership boundary easy to identify. It does not establish that offloading this tiny operation is worthwhile.

Measure the full Ruby call when evaluating user-visible latency. Separately measure worker startup, serialization, transfer and computation if you need to understand where the time goes. Comparing a warm GPU kernel with a Ruby call that includes all of those steps answers a different question.

When the worker needs to stay alive

A larger workload may need to reuse loaded weights or resident tensors. A long-lived worker can support that, but now the protocol needs request identifiers, deadlines, resource limits and recovery after partial failure. A crashed worker must not leave callers waiting forever or mix one request's reply with another's.

For a Ruby service, that is often a clearer next step than exposing raw CUDA pointers through FFI. A native C wrapper remains an option when its lifetime and error-handling costs are justified. The choice should follow the measured bottleneck and the library you need, not a fixed array-size rule.

The companion Python inference timing note examines the timing boundary inside a worker. Together, the two experiments separate a useful numerical operation from the machinery required to call it reliably.