Local LLMs: Buy a GPU or Rent One? RunPod at NT$24 an Hour, After Actually Trying It

Standing by the river envying the fish is no match for going home to weave a net.
—— Huainanzi, “Shuolin” (Western Han, c. 139 BC; translation mine)

What this is about

I’ve had a small nagging problem: many of the fun things you can build locally need a real graphics card, and this machine doesn’t have one. Last month I forced a batch job through the CPU once; it ran slower than real time. My conclusion then was to use cloud services first and keep the card money in my pocket.

Yesterday (2026-08-31) I got the other path working: not buying a card, but renting one by the second. The platform is RunPod; my friend Fred got the rental workflow running. I want to share how I see “rent versus buy” after actually doing it, and which of the common local-development uses belong on which side.

The short version: this is not a fight over performance. It’s a fight over usage.

Yesterday’s numbers first

One small batch job, four attempts (the first three were me stepping on rakes—more below), and the fourth ran clean. The four runs together used 567 seconds of GPU time; the bill was about NT$3.4 (RunPod dashboard, 2026-08-31). Per hour, a 4090 on Secure Cloud is US$0.74, about NT$24; Community Cloud is cheaper at US$0.34, about NT$11. The account had US$15 loaded, which at this rate is a few hundred runs.

Rent and buy: the shape of each path

Here are both sides laid out (prices from RunPod’s official pricing page, checked 2026-09-01; the card is estimated at about US$1,400 for a 4090-class card):

Up front: renting starts with a few dollars loaded; buying is ten-thousand-plus yuan, paid once.

Billing: renting is by the second, and killing the pod stops the meter; buying doesn’t pay anyone, but it costs electricity, depreciation, and the space on your desk. For a 4090, an hour rents for US$0.34 (Community Cloud) to US$0.74 (Secure Cloud); the per-hour depreciation of a card you own depends entirely on how many hours you run it.

Waiting: a rented pod takes one to three minutes to boot, and occasionally won’t connect and has to be swapped; your own card is always on.

The day it breaks: if the platform goes down, renters stop; your own machine doesn’t care about the network.

And the most interesting difference: renting, you can get an H100 or a B200 you could never afford; buying, you can’t afford them, and wouldn’t use them.

At the 4090 Community Cloud rate, ten-thousand-plus yuan buys roughly 4,100 hours. At two hours a day, breaking even takes more than five years; at eight hours a day, a year and a half. We all know what a five-year-old graphics card looks like.

So the question was never “which is cheaper.” It’s “which side of the crossover are you on.” I run this a handful of times a month, far from the crossover; renting is the obvious answer. Someone running ten-plus hours a day and generating images and video is on the other side.

When I wrote about local hardware last month, I described buying a card as insurance: it covers “the day everyone else raises prices or goes down, I can still work.” That still holds, with one addition: insurance is worth what your need for it is. Someone who hits an outage less than once a year is paying five years of premiums for it.

What RunPod can be used for

I only used its simplest form this time: start a machine, run a script, kill the machine. It actually comes in several shapes, suited to different jobs:

One: one-off batch work. Running the same processing over a whole batch of documents, upscaling a batch of old photos, or reading a few thousand screenshots in one go. Boot, run, kill, pay for those few minutes. This is the use I have no doubts about: Pods pricing.

Two: trying a card you can’t afford. Want to know what a model feels like on an H100? Rent one for two or three dollars an hour—more real than watching a review.

Three: fine-tuning. Training jobs run for hours to days and then end, exactly the shape of “rent.” RunPod containers can carry your own environment; download the weights when done.

Four: serverless endpoints. If you have a service that gets hit only occasionally (the backend of a small tool, say), you can deploy the model as a pay-per-call endpoint that costs nothing when idle. I haven’t run this one, only read the docs: Serverless overview.

Five: image and video generation. The generation services’ APIs bill per image and per second of output, and at volume the bill runs away fast. Renting a card and running an open model is a different matter: billed by the second, pay for what you run. I haven’t tried it yet; noting it here.

Tooling and tutorials are on the official docs site: docs.runpod.io.

Common local uses: which to rent for, which to buy for

Here are the things I’ve been doing myself lately, measured with the same ruler: does the job finish, or does it need to stay on?

Vector-indexing your own documents. Turn your notes, reports, and articles into vectors so one sentence finds the relevant passages later. I do this every day; today I recomputed 1,600-odd chunks on the CPU and it took half an hour. On a rented card it’s minutes, then kill the pod. This is the textbook “rent” shape: recompute occasionally, done when done. The usual models—EmbeddingGemma, BGE and the like—are small.

Batch reading of screenshots and images. A few thousand phone screenshots, images from chat logs, to be read in one pass and have their numbers and text pulled out. Done in batches, rent; if hundreds arrive every day and need handling as they come, that’s “stay on,” so decide by volume. Image-to-image search models like SigLIP and CLIP run fine locally.

A local model as your agent’s brain. General models like Qwen3.8 27B and Gemma 4 26B are one Ollama command away. If it runs long tasks for you every day—coding, sorting, organizing—that’s the “stay on” shape, buy a card; if you just want to feel it out, rent an hour.

Fine-tuning a small model of your own. Training runs for hours to days and then ends—exactly the “rent” shape. RunPod containers can carry your own environment; download the weights when done.

Image and video generation. ComfyUI with Stable Diffusion, Flux, LTX, MiniMax H3 and the like: rent a 48 GB card for batch output; buy if you generate every day.

Online models aren’t on this list. Something like DeepSeek V4 Flash, cheap and versatile, is neither rented nor bought—use the API.

One rule: only things that stay on and get used every day are worth buying; jobs that end, rent. The model’s name doesn’t matter. The usage does.

Three rakes I stepped on

I’m writing these down because RunPod’s official docs mention none of them; installing by the book doesn’t mean it runs.

First, the error message names the wrong culprit. The script told me “SSH public key is invalid,” with the correct file path attached. The key was fine. The real cause was that my machine didn’t have the small tool jq installed; the validation step returns non-zero as soon as it runs, so every key gets judged invalid. Lesson: when an error names something, verify that thing independently first, then look at what the code producing the message depends on.

Second, the direct network port fails to appear about half the time. That’s not a fault; it’s normal on RunPod’s Community Cloud. Fred’s script waits 180 seconds and swaps to another machine, up to three tries. Two of my first three rounds were spent waiting on this.

Third, always confirm the machine is dead when you’re done. If it isn’t, it’s still billing. After each round I went back to the RunPod site and checked that the Pods page was empty; all four times it was. This habit matters more than any money-saving trick.

One more, Windows-only: Git Bash silently rewrites paths like /root/... into Windows paths, which breaks the commands sent over SSH. The fix is to exclude just that one prefix; turning path conversion off entirely breaks file lookup for other local tools. Fred has folded this into the tool’s documentation.

Contributor to this piece: Fred

This article exists because my good friend Fred co-developed and tested the code. Booting the machine, connecting, sending the job up, killing the pod automatically when done, swapping to another machine when one won’t connect—he wrote that whole RunPod rental workflow as a script and got it running on his side first; every pitfall I hit on Windows went back into his docs. If you also want to wire RunPod into your own workflow, I recommend his GitHub: github.com/fredchu. No need to start from scratch.

One honest note: the same job doesn’t produce identical results in the cloud and locally. Fred measured it—about two percent apart, and nobody knows which side is more correct. The difference is stable, not noise. Know that before you use the cloud path.

References

• RunPod pricing: runpod.io/pricing (prices in this piece checked 2026-09-01; Community and Secure Cloud listed separately)
• RunPod docs: docs.runpod.io
• Fred’s GitHub: github.com/fredchu
• My earlier piece on local hardware: “Is a Local Qwen3.8 27B Right for You?” (2026-08-21); this one is its “rent” half
• Yesterday’s numbers: RunPod dashboard, 2026-08-31; the CPU comparisons: my own records from 2026-08-09 and 2026-09-01
• The opening line is from the Huainanzi; it came to mind when I saw the bill—this time I didn’t have to weave a net, someone by the river rents them

One thing to take with you

The line I kept after seeing the bill: renting versus buying isn’t about picking the cheaper one, it’s about which side of the crossover you’re on. I only let go of the idea of buying a card after working out “more than five years to break even.”

If you’re on the fence too, here’s a small thing I did that you might try: dig up the number of times last month you genuinely needed a GPU—not wanted to, actually did—and write that number on a piece of paper, with roughly how many hours each time next to it. Multiply the two; that’s your monthly usage. Once it’s written, you’re done.

If you want one more step, multiply those hours by US$0.34 and look at it next to the price of a card.

Raw Markdown