Most LLM prototypes start as a single Flask file and rot into a 2,000-line monster. Using flask blueprints llm application architecture from day one keeps routing, prompt logic, and provider integrations isolated so you can swap models without touching HTTP layers. This guide lays out an ordered path to refactor or bootstrap that separation.
1. Bootstrap with an application factory
Never instantiate Flask at module import time if you plan to use blueprints. Use a factory so you can register blueprints conditionally and inject configuration from environment or tests.
# app.py
from flask import Flask
from .chat import chat_bp
from .admin import admin_bp
from .llm_service import llm_bp
def create_app(config=None):
app = Flask(__name__)
app.config.from_mapping(
LLM_BASE_URL="https://api.openai.com/v1",
LLM_API_KEY=None,
DEFAULT_MODEL="gpt-4o-mini",
)
if config:
app.config.update(config)
app.register_blueprint(chat_bp, url_prefix="/v1")
app.register_blueprint(admin_bp, url_prefix="/admin")
app.register_blueprint(llm_bp)
return appThe factory pattern lets you mount the same flask blueprints llm application modules under different prefixes or skip admin in a serverless deploy. It also makes testing painless: build a minimal app with stub config and register only the blueprint under test.
2. Isolate the LLM client in a service blueprint
A blueprint is not just for routes. Use a blueprint to attach a shared client and helper functions to the app context. Keep the raw HTTP calls out of view functions.
# llm_service.py
from flask import Blueprint, current_app
from openai import OpenAI
llm_bp = Blueprint("llm", __name__)
def get_client():
# Reuse a client per app context
if "llm_client" not in current_app.config:
current_app.config["llm_client"] = OpenAI(
base_url=current_app.config["LLM_BASE_URL"],
api_key=current_app.config["LLM_API_KEY"],
)
return current_app.config["llm_client"]
def complete(prompt: str, model: str = None) -> str:
client = get_client()
model = model or current_app.config["DEFAULT_MODEL"]
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return resp.choices[0].message.contentThis keeps your chat routes thin. If you later route through a gateway such as n4n.ai, its OpenAI-compatible endpoint fronts 240+ models and automatic fallback when a provider is rate-limited or degraded, so the complete function only needs to forward a routing header instead of branching on provider SDKs.
Do not store the client as a module-level global. Flask’s app context is the correct scope; globals break under multiple apps in tests and cause cross-request leakage.
3. Split user-facing routes from internal ones
Create a chat_bp for end-user endpoints and an admin_bp for model config or usage inspection. Do not mix them; auth requirements differ.
# chat.py
from flask import Blueprint, request, jsonify
from .llm_service import complete
chat_bp = Blueprint("chat", __name__)
@chat_bp.post("/completions")
def completions():
data = request.get_json()
if not data or "prompt" not in data:
return jsonify(error="prompt required"), 400
try:
text = complete(data["prompt"], data.get("model"))
except Exception as e:
return jsonify(error=str(e)), 502
return jsonify(text=text)# admin.py
from flask import Blueprint, jsonify
from .llm_service import get_client
admin_bp = Blueprint("admin", __name__)
@admin_bp.get("/models")
def list_models():
client = get_client()
return jsonify([m.id for m in client.models.list().data])URL prefixes enforce boundaries at the routing layer. A flask blueprints llm application that mixes these invites accidental exposure of admin endpoints to the public internet. Put before_request auth hooks on admin_bp only.
4. Centralize model routing and cache hints
LLM calls need model selection, temperature, and cache-control. Put that in a config blueprint or a small routing.py used by the service layer. Avoid scattering model names across views.
# routing.py
ROUTING_MAP = {
"cheap": {"model": "gpt-4o-mini", "temperature": 0.2},
"smart": {"model": "gpt-4o", "temperature": 0.7, "cache": True},
}
def resolve_profile(name: str) -> dict:
return ROUTING_MAP.get(name, ROUTING_MAP["cheap"])In the service complete, accept a profile name. This decouples product logic (“use smart model for legal review”) from hard-coded strings.
def complete(prompt: str, profile: str = "cheap") -> str:
cfg = resolve_profile(profile)
client = get_client()
extra = {}
if cfg.get("cache"):
extra["headers"] = {"Cache-Control": "max-age=3600"}
resp = client.chat.completions.create(
model=cfg["model"],
messages=[{"role": "user", "content": prompt}],
temperature=cfg["temperature"],
**extra,
)
return resp.choices[0].message.contentIf your gateway honors client routing directives and forwards provider cache-control hints, those headers are respected downstream without extra code. The blueprint stays ignorant of which vendor serves the token.
5. Streaming and async boundaries
Flask is synchronous. LLM streaming via Response with a generator works, but blocks the worker thread. For a flask blueprints llm application under load, offload to a task queue or use an async gateway.
@chat_bp.post("/stream")
def stream():
def gen():
client = get_client()
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello"}],
stream=True,
)
for chunk in stream:
yield chunk.choices[0].delta.content or ""
return current_app.response_class(gen(), mimetype="text/plain")Tradeoff: synchronous streaming ties up a WSGI worker per connection. If you expect concurrent users, put streaming behind a dedicated blueprint mounted on a separate worker class or use WebSocket via a different ASGI app. Do not pretend a blueprint makes Flask async; it does not.
6. Test blueprints in isolation
Because blueprints are registered in the factory, you can test each with a minimal app.
import pytest
from .app import create_app
@pytest.fixture
def app():
return create_app({"LLM_API_KEY": "test", "TESTING": True})
def test_completions_missing_prompt(app):
client = app.test_client()
resp = client.post("/v1/completions", json={})
assert resp.status_code == 400Mock the complete function with monkeypatch to avoid real API calls. This isolation is the main payoff of the blueprint approach: you test routing and validation without booting the whole LLM stack. Test the admin blueprint separately with a forbidden assertion when auth is missing.
7. Common pitfalls and tradeoffs
Circular imports. Putting from .llm_service import complete at the top of chat.py while llm_service imports current_app is fine, but if llm_service imports chat_bp you’ll loop. Keep dependency direction one-way: blueprints depend on service, not vice versa.
Shared client state. Storing the LLM client in app.config works for single-process dev. Under gunicorn with multiple workers, each worker gets its own client—fine, but don’t assume in-memory rate limit counters are global. Use the gateway’s per-token usage metering if you need cross-worker accounting.
Blueprint overuse. Don’t make a blueprint per model. Blueprint is for cohesive route groups, not for every configuration variant. Use the routing map from section 4.
Error handling. A flask blueprints llm application should define a blueprint-level error handler for 502 from provider timeouts, but avoid catching all exceptions silently. Let unexpected errors surface to Sentry.
Prefix collisions. Registering two blueprints with the same url_prefix and overlapping routes fails silently until a request hits the wrong handler. Use app.url_map inspection in tests to assert expected routes.
8. Deploy checklist
- Register blueprints with explicit
url_prefixto avoid route collisions. - Load
LLM_BASE_URLfrom env; never hard-code in blueprints. - Use
app.configfor client singleton, not module globals. - If using a gateway, set
extra_headersfor routing in the service layer, not in views. - Add a health route in a
system_bpthat checks client connectivity without consuming tokens.
Following this ordered path keeps your LLM integration testable and lets you swap providers by changing one config key. The blueprint structure pays off the moment you add a second model, a background job, or a partner-facing API.