How to Share Ollama Server Through IP Address and Port

Introduction

Recently, I was configuring a quick server to run LLM locally. Though it is easy in hindsight, it was not as clear-cut as it needed to be. After troubleshooting and following the Official Ollama FAQ, I found a solution by configuring the OLLAMA_HOST environment variable. Here's a step-by-step guide on how to resolve this problem.

The Issue

By default, Ollama binds to 127.0.0.1 on port 11434, which restricts access to the local machine. This means that even if you specify an IP address and port, other devices on the network will not be able to access the Ollama server.

Solution: Configuring OLLAMA_HOST

Again, it is really easy and not worth even talking about. Nonetheless, to make Ollama accessible from other devices on your network, you need to change the bind address from 127.0.0.1 to 0.0.0.0, which allows the server to listen on all available IP addresses. Follow these steps to configure OLLAMA_HOST:

Step 1: Set OLLAMA_HOST Environment Variable

You need to set the OLLAMA_HOST environment variable to 0.0.0.0:11434. The process for setting environment variables varies depending on your operating system:

On macOS

1. Open the Terminal app.
2. Use launchctl to set the environment variable:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"

3. Run Ollama as a server:

ollama serve

4. Verify the environment variable is set correctly by running:

echo $OLLAMA_HOST

It should display 0.0.0.0:11434.

Make sure you exit the Ollama app before starting it as a server.

On Linux

1. Edit the systemd service by calling systemctl edit ollama.service.
2. Add the following line under the [Service] section:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

3. Save and exit the editor.
4. Reload systemd and restart Ollama:

sudo systemctl daemon-reload
sudo systemctl restart ollama

On Windows

1. Quit the Ollama application by clicking on it in the taskbar.
2. Open the Settings (Windows 11) or Control Panel (Windows 10), and search for Environment Variables.
3. Click on Edit environment variables for your account.
4. Add or edit the variable for OLLAMA_HOST and set its value to 0.0.0.0:11434.
5. Click OK/Apply to save the changes.
6. Restart the Ollama application from the Start menu.

Step 2: Verify the Configuration

After setting the environment variable, you can check if the Ollama server is accessible from another device on the network.

1. From another device, open a browser or use a tool like curl to access the server:

http://your_ip_address:11434

2. If everything is configured correctly, you should see a response from the Ollama server.

Additional Tips

• Firewall Settings: Make sure that the firewall on your host machine allows incoming connections on port 11434.
• Port Forwarding: If you are accessing the server from a different network, you may need to configure port forwarding on your router.

Conclusion

By following these steps, you can easily share your Ollama server with other devices on your network. This is particularly useful for collaborative projects or when you want to access the server from multiple devices.

For more details, here is the Official FAQ.