Building AI Knowledge Systems Inside Federal Boundaries

There is a version of federal AI adoption that exists mostly in slide decks: pick a model, connect the data, transform the mission. Anyone who has actually operated inside a federal boundary knows the real work looks different. Over the past year we built and now operate several AI knowledge systems for federal health contexts, and the lessons that mattered were not the ones the industry talks about.

The boundary is the design input. Federal health work comes with constraints that do not negotiate: data that cannot leave certain environments, systems that must trace every claim to a source, and users who are accountable for what they act on. We stopped treating these as limits on what AI could do and started treating them as the specification. Retrieval grounded in an approved corpus, citations on every answer, a system that says plainly when it does not know: these are not compliance features bolted on at the end. They are what makes the system usable by people whose decisions carry consequences.

Cited answers changed adoption more than model quality did. When we put a knowledge system over a public federal research portfolio, the feature that earned trust was not fluency. It was that every response linked back to the government’s own pages, so a skeptical reviewer could check any claim in one click. Federal staff do not need AI to sound confident. They need it to be checkable.

Exact numbers require different machinery than good prose. Ask a language model how many projects in a portfolio use machine learning and it will estimate. Estimates are poison in a federal context, where someone will repeat that number in a briefing. We learned to route counting questions to computation over the complete dataset and reserve the language model for what it is actually good at: synthesis, explanation, and summary with sources. The distinction sounds technical. It is the difference between a demo and a tool.

On-premises AI is now a real option, not a compromise. We run open-weight models on hardware in our own lab, inside our own boundary, at speeds that support daily work. The frontier models are better, and for some tasks the difference matters. But for a growing class of federal use cases, the question is not which model is smartest. It is which model can operate where the data is allowed to live. That answer changes monthly, which is why we keep testing and publishing what we find.

Start where the data is already public. The fastest path we have found to demonstrating value is to build over content an agency has already published: research portfolios, guidance libraries, program documentation. No new authorities, no data agreements, no risk. If the approach proves out there, the harder conversations about internal data start from evidence instead of promises.

None of this is a technology story, in the end. It is an operating discipline: connect approved sources, keep humans in review, make every claim traceable, and measure whether the system is actually right. We wrote that discipline down in our white paper on AI-enabled knowledge infrastructure, and we run our own company on systems built the same way, because recommending an approach you do not live with yourself is another thing federal staff can spot in one click.

Daniyal Syed is the founder and CEO of Conselara, a federal health IT firm working with HHS agencies on cloud, AI, and digital health delivery.