Tag

#ops

2 articles tagged with "ops"

#inference5 #model-hosting4 #grace-hopper3 #serverless-gpu3 #platform2 #cuda2 #ops2 #gpu2 #pricing2 #gpu-cloud2 #ion1 #ionattention1 #gpu-kernels1 #model-serving1 #router1

March 7, 20264 min read

What is an Inference Platform?

Inference is not just a model call. It is routing, caching, observability, evaluation, fine-tuning, and the runtime that executes the call. A complete guide.

guideplatforminference+1

Read article

February 11, 20263 min read

Continuous Evaluation: Beyond Judge Consensus

LLM-as-judge with majority vote is the default — and it is wrong often enough to lose trust. A better approach combines synthetic data, deterministic heuristics, judges, and shadow evaluation on live traffic.

evaluationllm-judgeshadow-evaluation+1

Read article