Your Cheapest AI Model Might Be Your Most Expensive Workflow

A lower generation price can lose its advantage when people spend more time checking and correcting the output. Before choosing an AI model for a business workflow, measure the work that happens after generation as well as the API usage.

This guide supplies a small calculator and a trial record you can reuse. The arithmetic is executed and tested. The candidate inputs are deliberately fictional: no real models were compared, no vendor prices were sampled and no human review study was performed.

Start with a task and a quality gate

Fictional scenario: Orbit Studio prepares project-summary drafts. Its acceptance rule is that the summary must preserve supplied scope, identify unanswered questions and avoid inventing a delivery promise. Cost only matters after the workflow meets its required quality and risk criteria.

OpenAI’s evaluation guidance recommends task-specific tests, representative cases, human-calibrated metrics and continuing evaluation as the system changes. Use that structure for your own workflow rather than choosing from a generic leaderboard alone. Evaluation best practices.

Define what counts as review. A mandatory glance at every draft is still review time, even when nothing changes. Include it in the measured per-job workload or account for it separately; do not quietly drop it because a model appears accurate.

The arithmetic behind the decision

For this narrow calculator: generation cost = jobs × generation cost per job. Expected review hours = jobs × review fraction × minutes per reviewed job ÷ 60. Modeled total = generation cost + review hours × hourly review cost.

Generation alone contributes USD2 for A and USD8 for B. Under these invented assumptions, review contributes USD450 and USD75 respectively. That illustrates why the ordering can reverse; it does not establish that any more expensive model needs less review or that either workflow is fit to deploy.

Run the calculator

Save this as workflow_cost.py. It needs Python only; this run used Windows and Python 3.14.3. Pass numeric assumptions as decimal strings. Python’s decimal module supports decimal arithmetic and explicit quantization/rounding; this example uses half-up rounding for displayed totals. It is a planning calculation, not an invoice engine. Python decimal documentation.

"""Illustrative workload arithmetic, not vendor pricing or a model benchmark."""
from decimal import Decimal, InvalidOperation, ROUND_HALF_UP
import json


def number(value, maximum):
    if type(value) is not str or len(value) > 40:
        raise ValueError('Use short decimal strings')
    try:
        result = Decimal(value)
    except InvalidOperation as exc:
        raise ValueError('Invalid decimal') from exc
    if not result.is_finite() or not 0 <= result <= Decimal(maximum):
        raise ValueError('Decimal outside allowed range')
    return result


def cost(jobs, generation_per_job, review_fraction, minutes_per_review, hourly_cost):
    if type(jobs) is not int or not 0 <= jobs <= 1_000_000:
        raise ValueError('jobs must be an integer from 0 to 1000000')
    generation = number(generation_per_job, '1000')
    fraction = number(review_fraction, '1')
    minutes = number(minutes_per_review, '1440')
    hourly = number(hourly_cost, '1000')
    hours = Decimal(jobs) * fraction * minutes / Decimal(60)
    generated_cost = Decimal(jobs) * generation
    review_cost = hours * hourly
    def display(value):
        return str(value.quantize(Decimal('0.01'), rounding=ROUND_HALF_UP))
    return {'generation_cost_usd': display(generated_cost),
            'expected_review_hours': display(hours),
            'review_cost_usd': display(review_cost),
            'modeled_total_usd': display(generated_cost + review_cost)}


if __name__ == '__main__':
    print(json.dumps({'fictional_A': cost(1000, '0.002', '0.30', '3', '30'),
                      'fictional_B': cost(1000, '0.008', '0.05', '3', '30')}, indent=2))

The helper bounds job count, costs, review fraction and review minutes. It rejects booleans as job counts, ordinary numeric objects where decimal strings are required, negative inputs and non-finite values. Those bounds are example constraints, not recommended limits for every business.

Expected review hours are an average workload calculation, not a staffing schedule. A fractional expected review count can be meaningful for a forecast; an actual batch has discrete jobs. The hourly value can represent an explicitly chosen opportunity-cost assumption, not necessarily new cash spending.

Verify the arithmetic before filling in your own data

Save the tests as test_workflow_cost.py:

import unittest
from workflow_cost import cost


class CostTests(unittest.TestCase):
    def test_a(self):
        self.assertEqual(cost(1000, '0.002', '0.30', '3', '30'),
                         {'generation_cost_usd': '2.00', 'expected_review_hours': '15.00',
                          'review_cost_usd': '450.00', 'modeled_total_usd': '452.00'})

    def test_b(self):
        self.assertEqual(cost(1000, '0.008', '0.05', '3', '30')['modeled_total_usd'], '83.00')

    def test_zero_jobs(self):
        self.assertEqual(set(cost(0, '1', '1', '3', '30').values()), {'0.00'})

    def test_no_review(self):
        self.assertEqual(cost(10, '0.1', '0', '3', '30')['modeled_total_usd'], '1.00')

    def test_all_review(self):
        self.assertEqual(cost(60, '0', '1', '1', '30')['expected_review_hours'], '1.00')

    def test_bad_fraction(self):
        with self.assertRaises(ValueError): cost(10, '1', '1.01', '3', '30')

    def test_negative(self):
        with self.assertRaises(ValueError): cost(10, '-1', '0.1', '3', '30')

    def test_nonfinite(self):
        for value in ('NaN', 'Infinity', '-Infinity'):
            with self.subTest(value=value):
                with self.assertRaises(ValueError): cost(10, value, '0.1', '3', '30')

    def test_numeric_type(self):
        with self.assertRaises(ValueError): cost(10, 0.1, '0.1', '3', '30')

    def test_bool_jobs(self):
        with self.assertRaises(ValueError): cost(True, '1', '0.1', '3', '30')

    def test_round_half_up(self):
        self.assertEqual(cost(1, '0.005', '0', '0', '0')['modeled_total_usd'], '0.01')


if __name__ == '__main__':
    unittest.main()

Run:

python -X utf8 -m unittest -v test_workflow_cost.py
python -X utf8 workflow_cost.py

Eleven test methods passed. They verify the two synthetic examples, zero jobs, no/all review, invalid fraction, negative cost, non-finite values, numeric type restrictions, boolean rejection and half-up rounding. The sample output was:

{
  "fictional_A": {
    "generation_cost_usd": "2.00",
    "expected_review_hours": "15.00",
    "review_cost_usd": "450.00",
    "modeled_total_usd": "452.00"
  },
  "fictional_B": {
    "generation_cost_usd": "8.00",
    "expected_review_hours": "2.50",
    "review_cost_usd": "75.00",
    "modeled_total_usd": "83.00"
  }
}

The final article’s program and tests were extracted into a disposable directory and the documented commands were rerun. Output matched this sample. That checks the calculator, not the assumed review fractions.

Replace assumptions with a fair task trial

Use the same representative tasks, source material and acceptance rubric for each candidate. Include common cases, missing information and cases that would create an unacceptable business promise. Keep model identifiers, prompt revisions, settings and retrieval/tool inputs with the results. If a workflow uses different scaffolding, describe the comparison as workflows rather than attributing every difference to the model alone.

For each task, record: task ID, candidate/version, attempt count, observed generation/tool charges, quality outcome, reviewer minutes, final accepted/rejected state and material failure reason. Review outputs without candidate labels where practical. Keep rejected jobs and retries in the record; removing them makes the apparent cost of accepted work look better than it is.

The calculator has one generation-cost-per-job input. Include the relevant retry/tool charges in that input or extend the calculation explicitly. If every task receives initial review plus only some need correction, either calculate both stages separately or use a clearly defined effective review-time input. Do not double-count the same reviewer time.

For business-weighted evaluation rather than accuracy alone, see catching routing regressions that accuracy misses. The cost worksheet complements a quality gate; it does not replace it.

Check capacity and uncertainty, not just the displayed total

Even a workflow with an acceptable modeled total may be unusable if review demand exceeds available reviewer hours. Compare expected hours with actual capacity and the timing of arrivals. This calculator does not model queues, peaks or service levels.

Run sensitivity cases with worse review fractions and longer corrections before relying on a single point assumption. Small trials can miss rare costly errors. Recheck after a meaningful model, prompt or data change. A cheaper output that makes an unauthorized promise does not become acceptable because its arithmetic total is low.

No real model, API billing, review-time measurement, queue, production integration or business saving was tested here. The calculation omits setup, monitoring, incident cost and any charges you did not put into the inputs. It is not a full total-cost forecast or a recommendation to buy a specific model. The cover is generated conceptual artwork; the interior table is synthetic planning data with executed arithmetic.

Jason Ead publishes AlgoThoughts AI News and connects practical AI decisions with My Biz Heroes: websites, automation, workflows, AI tools and training. Explore Jason’s work. Price the workflow you can actually operate, including the people who must review its output.