Loaded Tokens. The latest AI news in five minutes or less.Get the briefing
Learn / Models & how they differ

Open weights are not a free lunch with a GPU garnish

Separate access to model files from licenses, hosting costs and operational ownership.

The shape of itConceptual workflow, not a benchmark or a measured result. Read with the linked sources below.
  1. Model files + license
  2. Choose runtime
  3. Operate or rent
  4. Evaluate the application

Access is not the same as permission

“Open” can mean several different things. A model may publish downloadable weights while limiting how they may be used. Training data, training code and the model license are separate questions. The Hugging Face course introduces the ecosystem; individual model cards and licenses are the authoritative place to check a specific model. Do not assume “available to download” means “unrestricted commercial use.”

A hosted API keeps the runtime behind a service interface. A self-hosted model puts more of the deployment burden on you. That is not a moral ranking. It is a division of responsibility. Our recommendation is to choose based on data needs, available expertise and predictable workload, not on which phrase sounds more independent.

The ownership checklist

For a hosted service, inspect provider terms, retention, access controls, limits and the cost of the selected model. For self-hosting, add hardware capacity, runtime configuration, updates, monitoring and incident response. Neither path removes the need to evaluate the outputs. This application-level risk view follows the spirit of the NIST AI Risk Management Framework.

Do not compare “free weights” with “paid API” and declare the spreadsheet finished. Compute, storage, networking, staff time and idle capacity belong in the self-hosted column. Token charges, tools and surrounding infrastructure belong in the API column. The OpenAI rate card illustrates the multiple components of one hosted provider’s bill; it is not a quote for operating another model.

When each route deserves a look

Our editorial take: consider a hosted service when you want a quick, maintained interface and its terms fit your data. Consider self-hosting when you have a specific control requirement and the capacity to operate the system. A local demonstration on a laptop does not prove that a multi-user production workload will fit. A managed service’s security marketing does not replace your own access review.

Keep portability concrete. Can you export prompts, evaluation cases and application data? Which parts depend on a provider-specific feature? An abstraction layer can help organize code, but differences in models and tools still need verification. Own your acceptance tests even if you rent the runtime.

Read a model card like a buyer

Exercise: A repository includes model files, a usage license and hardware guidance. What should you record before adoption?

Answer: The exact model revision, permitted uses, restrictions, required runtime resources, intended task, known limitations and your evaluation results. If a license is unclear, get qualified advice rather than letting a chatbot invent a permission slip.

The takeaway: downloadable is an access property. Open-source licensing is a legal property. Running reliably is an operational job. Put them on separate lines and the decision gets less ideological—and more useful.

Sources

This explainer is built from public sources. We didn't test or measure anything ourselves. The examples and diagrams are simplified sketches.

A twice-a-week note, if you want it.