Why we built SecondStack

“Why build another enterprise AI platform? Isn’t this solved by now?”

Fair question. When we needed to give every team in our own company governed access to AI, none of the three usual paths held up. SaaS assistants park your data in a vendor’s cloud and bill per seat. Self-assembled open-source stacks leave you running an unsupported platform that breaks at the seams. And doing nothing gets you shadow AI, with staff pasting confidential data into accounts that may train on it.

So we built a fourth path, and ran it on ourselves first.

The three paths, and where each one breaks

Buy a SaaS assistant. ChatGPT Enterprise, Claude Enterprise, and Copilot are great products, but your full chat histories and attached files come to rest in the vendor’s cloud, under their retention and access controls rather than yours. For a regulated organization that’s the whole conversation: you inherit the vendor’s leakage risk on your accumulated data, you’re locked into one provider’s pricing and roadmap, and a policy change or account suspension can cut off the tool your company now runs on, with no fallback. Pricing is per seat, too: one mid-size enterprise we spoke with was quoted roughly $500K a year just to roll a frontier assistant out company-wide.

Assemble it yourself. An API gateway like LiteLLM, a chat UI like LibreChat or OpenWebUI, plus guardrails, agents, observability, monitoring, and so on. Everything has a few open-source offerings, so it sounds doable, and plenty of teams try it.

The caveats are subtle at first. The open-source “guardrails” modules look like they work, but they miss real risks while still tripping on every other benign query; the serious, high-quality options sit behind separate enterprise agreements and are typically SaaS. Getting a chat UI to enforce a separate budget per user turns out to be nontrivial. SSO mapping ends up spread across three places. The “observability” module shows charts that barely make sense and can’t break spend down by the projects your users are actually working on. And then the whole thing, which seemed to work, quietly falls apart by month two.

Do nothing. Officially, anyway. Employees don’t wait: they paste work into personal ChatGPT accounts, where consumer tiers may train on whatever they submit, and IT sees none of it. Many of the companies we talk to are here without ever having chosen it.

Where SecondStack came from

As a consulting firm, T1A has helped enterprises adopt data, and more recently AI, for a living. When the GPT-4-era models arrived, the efficiency they unlocked was impossible to ignore, and we wanted that leverage in the hands of every team, not just a few engineers running experiments on the side. The catch was the nature of our work. We handle sensitive client materials under strict confidentiality, so simply subscribing everyone to chatgpt.com or claude.ai was not on the table. Whatever we rolled out had to keep that data on our side of the fence.

So we set out to build an in-house platform. The first attempt did the obvious thing: stitch together the well-known open-source titles, OpenWebUI, the LiteLLM gateway, and assorted add-ons. It got us off the ground, but the limitations showed up quickly, exactly the seams described above. The pieces didn’t stack cleanly, budgets and identity fought us at every turn, and the parts that looked finished quietly came apart once real traffic hit them.

The first internal version of what became SecondStack went into production in 2025. From day one, people ran real work through it and came back with the gaps, and the platform was shaped by that traffic rather than by a roadmap written in a vacuum.

What we actually needed

As the operators of that platform, our list was specific:

• A modern, powerful chat UI for every user persona, with access to all frontier models (Claude, GPT, Gemini, and locally served open models).
• Agentic muscle, not just Q&A: reason over uploaded documents, run tools, write and execute code, and build ad-hoc artifacts on the spot, e.g. a spreadsheet, a chart, a slide draft, or an app.
• Seamless connections to other systems and resources via Skills, MCP, and built-in Connectors.
• Collaboration on Projects with RAG, Knowledge Collections, Memory, and Group Chats.
• An OpenAI- and Anthropic-compatible API gateway with virtual keys, so IDEs, scripts, and local coding agents get the same governed access as human users.
• Budgets and multi-stage spend alerts per user, team, and key, with hard cutoffs, so a runaway agent hits its own limit instead of locking out the team.
• Guardrails that strip PII and secrets on the way out and unmask them on the way back in.
• RAG over our own documents, plus SSO with group sync from the IdP.
• All user data stored locally, in our own PostgreSQL and S3.

And nothing priced per seat: usage simply scales with LLM API consumption, billed by the providers to accounts we control. No single product covered that list.

What SecondStack is, and what it isn’t

SecondStack is a self-hosted AI platform, bundling chat, agents, an LLM API gateway, guardrails, cost controls, and observability, running on infrastructure you control, with all your data (chat history, attached documents, agentic sessions, and usage logs) kept in your own database.

SecondStack bundles our first-class software with curated and robust open-source components we stand behind, and we support the whole stack as a single point of contact. Deployment is deliberately boring: Docker Compose on a small Linux VM, or Kubernetes when your platform team requires it.

It isn’t SaaS: you host it, or we run a single-tenant instance for you. It isn’t just a gateway; that’s one component of the bundle. And it isn’t another chat UI, because the chat sits on top of the governance layer rather than instead of it.

If you’re inside that trilemma right now, write to hello@secondstack.ai. We’ll show you what we run, including the parts that were harder to get right than we expected.