Quilyx

Digital sovereignty

The AI you plug into the wall. What local AI gives a client

Local AI is AI that you plug into the wall, own, and control. That is how Alec Fong described it at NVIDIA GTC in Berlin. Fong leads NVIDIA Brev and was the co-founder and CTO of Brev, which NVIDIA acquired in 2024. His session was called Data Center Under Your Desk. His team's data center stands in his living room.

In this blog, we look at what his team runs in-house and what it sends to the cloud, what that delivers, what it costs, and where it is still rough, with the questions a client can ask along the way.

What local AI is

Local AI runs on a device that you own. That can be a PC that runs a model, an extra graphics card next to your own device, or a workstation that serves a whole team.

Fong's team has about twenty software developers. They started with two small machines under his desk, about ten thousand dollars for the pair. Two workstations with data-center-grade chips followed, behind his couch. Those now serve the whole team.

Open models make this possible. These are models that anyone can download and run. According to Fong, they trail the closed models of the large labs by three to five months, and that gap has been remarkably constant. Because they are open, they are also adapted to smaller machines. So performance on your own equipment follows the frontier with a delay of a few months.

The thinking in the cloud, the execution in-house

Fong's team uses both. The models in the cloud do the deep thinking, such as planning and research. Local agents, programs that carry out tasks independently, do the execution. They build, test, and iterate. NVIDIA released an open tool that chooses per task which model does the work.

That division brings privacy with it. The local agents work with the sensitive data, so the models in the cloud have no need for it.

Fong prefers to keep production logs, which can contain sensitive customer data, away from an outside party when a failure has to be traced. On his own equipment, an agent does that work. His slide listed more work of this kind.

  • tracing failures in logs that contain customer data
  • reproducing customer issues from private support conversations
  • analyzing tax returns, bank statements, and passport scans
  • drafting legal documents from confidential correspondence

Part of the team's work has run on local models since the end of July. That share is now about a fifth, and Fong expects it to grow strongly in the coming months.

For a client, the question is which work has to stay inside and which work may go to an outside service.

What it delivers

Fong named the advantages of owning the equipment.

  • Privacy. Whoever controls the hardware and the model has a simple security perimeter. The data stays inside.
  • Availability. The model runs on a device you own and keeps working when the internet goes down.
  • Speed. The model sits close to the work. For a robot, a delay across thousands of kilometers is too much.
  • Peace of mind about the bill. Whoever owns the hardware has, in effect, unlimited use. With an outside service, every question counts toward the invoice.
  • Certainty about the service. The maker of a closed model controls the execution and can downgrade a user. In critical security work, where minutes matter, that is a risk.

With the local models added, the team finished more than it could have otherwise, including the replacement of a database and a new part of the product.

Because the hardware is already paid for, work that used to be too expensive can now be added. An agent searches around the clock for breaches and weak spots. According to Fong, the question in the team shifts from what AI costs to what the team can do with it.

What it costs

Whoever buys their own equipment invests up front and then saves on the use of outside services. On top of that comes what Fong called the sovereignty premium. That is what you pay for privacy, reliability, and speed, also while the investment still has to earn itself back.

These are the amounts Fong showed.

| Item | Amount | |---|---| | one small machine | from about 5,000 dollars | | two workstations, as his team has | about 200,000 dollars | | extra power and cooling at his home | about 500 dollars a month | | saving compared with the same open models at a provider | about 11,000 dollars a month | | payback period of the workstations | about a year and a half |

If the same use comes off closed models, Fong puts the saving at about 100,000 dollars a month.

The saving assumes machines that are busy almost all day, and Fong called that high. Depreciation of the hardware was left out, and the lifetime is hard to estimate. He does see older graphics cards hold their value longer than expected, because new models keep running better on existing hardware.

The question for a client is whether these items are in the business case, with the sovereignty premium as a line of its own. Hardware is an investment with a payback period, where a subscription is a monthly bill.

Where it is still rough

Running your own machines is young. Fong said there are still many sharp edges and that it took more work than he had expected. An agent that patches the machines and keeps them up to date takes over part of that work.

Capacity is hard to plan as well. One user can now start thirty agents at once, so the number of users says little about the load. According to Fong, more people still fit on the same machine than you would think, because colleagues rarely ask for full power at the same moment.

At the start of this year, according to Fong, the choice was to let an agent do everything unsupervised or to approve every action yourself. His team now runs agents in a sandboxed environment, with a policy set in advance for the files, programs, and websites an agent may reach.

The question for a client here is who runs the machine, and whether that knowledge is in-house.

The same at an engineer's desk

A second session, which we went through afterward, showed what a machine at the desk does in an industrial company. Richard Borge leads a team for AI and accelerated computing at Baker Hughes, an energy technology company. In The Deskside Advantage, a session by Dell, he spoke about designing compressors.

A flow simulation of one design variant takes twelve hours there, and a new design can require thousands of variants. Borge's team built an AI model that predicts the outcome of such a simulation in a tenth of a second. The model was trained on 52 simulations. On three designs that were new to the model, it reached a median accuracy of 99.5 percent. The simulation keeps the final say on the design that is chosen.

The full model fits on a Dell workstation. According to Dell, everything runs on the company's own premises and under its own control. Borge called the research still early.

What this means for an AI strategy

Where AI runs is a strategy choice. It decides what an organization owns and what it rents, and it touches governance, data, people, and suppliers. An AI strategy answers the questions in this blog, which are what stays inside, whether the business case carries the sovereignty premium, and who runs the machines.

Quilyx advises on AI strategy and takes AI ambitions from the boardroom to working results. Sovereign AI is our starting point, because sensitive data belongs on own hardware, under own control. Where a hosted service is good enough, we say so too. We advise from our own practice, since we build and run AI systems ourselves, on our own hardware in the Netherlands.

The use of AI in Fong's team went from 2.6 billion tokens a month in January to more than 47 billion in September. Tokens are the pieces of text in which the use of AI is counted. The local models joined at the end of July, and their share has grown every month since. So the movement is young. Whoever starts now is, in our view, ahead.

About this article: This article is our own reading of two sessions at NVIDIA GTC Berlin in October 2026, Data Center Under Your Desk and The Deskside Advantage. The ideas from those sessions belong to the speakers and are credited to them. The translation to clients, the positions and the choices in this article are ours. We wrote and edited it together with AI, which helped with the research, the working out and the wording. Quilyx carries the editorial responsibility.

Do you want something?

Tell us what you want to achieve. We help make it clear and take it to a working result.

info@quilyx.com or +31 30 401 0422