Set up and run Azure Foundry Native LLM server & Open WebUI on Home windows Server 2025

Foundry Native is an on-device AI inference answer providing efficiency, privateness, customization, and price benefits. It integrates seamlessly into your present workflows and functions by way of an intuitive CLI, SDK, and REST API. Foundry Native has the next advantages:

• On-Machine Inference: Run fashions regionally by yourself {hardware}, lowering your prices whereas retaining all of your knowledge in your machine.
• Mannequin Customization: Choose from preset fashions or use your personal to satisfy particular necessities and use circumstances.
• Value Effectivity: Eradicate recurring cloud service prices through the use of your present {hardware}, making AI extra accessible.
• Seamless Integration: Join together with your functions by way of an SDK, API endpoints, or the CLI, with straightforward scaling to Azure AI Foundry as your wants develop.

Foundry Native is good for eventualities the place:

• You need to maintain delicate knowledge in your machine.
• It’s good to function in environments with restricted or no web connectivity.
• You need to cut back cloud inference prices.
• You want low-latency AI responses for real-time functions.
• You need to experiment with AI fashions earlier than deploying to a cloud setting.

You’ll be able to set up Foundry Native by operating the next command:

winget set up Microsoft.FoundryLocal

As soon as Foundry Native is put in, you obtain and work together with a mannequin from the command line through the use of a command like:

foundry mannequin run phi-4

It will obtain the phi-4 mannequin and supply a textual content based mostly chat interface. If you wish to work together with Foundry Native by way of an internet chat interface, you should use the open supply Open WebUI mission. You’ll be able to set up Open WebUI on Home windows Server by performing the next steps:

Obtain OpenWebUIInstaller.exe from https://github.com/BrainDriveAI/OpenWebUI_CondaInstaller/releases. You will get warning messages from Home windows Defender SmartScreen. Copy OpenWebUIInstaller.exe into C:Temp.

In an elevated command immediate, run the next instructions

winget set up -e --id Anaconda.Miniconda3 --scope machine
$env:Path="C:ProgramDataminiconda3;" + $env:Path
$env:Path="C:ProgramDataminiconda3Scripts;" + $env:Path
$env:Path="C:ProgramDataminiconda3Librarybin;" + $env:Path
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/fundamental
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/r
conda.exe tos settle for --override-channels --channel https://repo.anaconda.com/pkgs/msys2
C:TempOpenWebUIInstaller.exe

Then from the dialog select to put in and run Open WebUI. You then must take a number of additional steps to configure Open WebUI to connect with the Foundry Native endpoint.

1. Allow Direct Connections in Open WebUI
   1. Choose Settings and Admin Settings within the profile menu.
   2. Choose Connections within the navigation menu.
   3. Allow Direct Connections by turning on the toggle. This permits customers to connect with their very own OpenAI suitable API endpoints.
2. Join Open WebUI to Foundry Native:
   1. Choose Settings within the profile menu.
   2. Choose Connections within the navigation menu.
   3. Choose + by Handle Direct Connections.
   4. For the URL, enter http://localhost:PORT/v1 the place PORT is the Foundry Native endpoint port (use the CLI command foundry service standing to search out it). Be aware that Foundry Native dynamically assigns a port, so it is not at all times the identical.
   5. For the Auth, choose None.
   6. Choose Save

➡️ What’s Foundry Native https://study.microsoft.com/en-us/azure/ai-foundry/foundry-local/what-is-foundry-local

➡️ Open WebUI: https://docs.openwebui.com/