Ollama: The Ultimate Guide

Ollama is a powerful, open-source tool that helps you run large language models (LLMs) on your computer. This means you can build and test AI tools without needing an internet connection or expensive cloud subscriptions. It's a game-changer for people who work in software development and quality assurance.

The main reason QA professionals love Ollama is because it keeps your data private and secure. By running AI models on your own machine, your confidential information never leaves your computer. This makes it a perfect solution for testing and building secure local AI applications.

Choosing between local and cloud AI is a major decision for any QA professional. While cloud solutions offer simplicity and massive scalability, a local setup gives you complete control. This comparison highlights why running models locally with Ollama is a powerful option for your daily work, especially for testing, security, and cost control.

Ollama's library is home to dozens of powerful models, each with different strengths. The model you choose will depend on your specific needs and hardware. This guide helps you quickly find the right one for you.

When choosing a model, two specifications are the most important to consider for performance and capability: the number of parameters and the context window. Use the expandable panels below to learn about each.

The command-line interface (CLI) is a powerful tool for rapid testing and managing your local environment. The REST API, which we'll discuss next, is the other key way to automate your workflows.

Comprehensive Command Reference

The true power of Ollama for QA automation comes from its REST API. You can send requests to your local Ollama server from any programming language or testing tool to integrate LLMs directly into your test suites and applications. This allows you to perform tasks like automated test case generation, bug report summarization, and more, all without leaving your existing workflow.

REST API Endpoints

Most Ollama CLI commands can be performed via the REST API. This is the foundation for building automated workflows, custom frontends, and integrations with your existing tools.

import requests
import json

# The local Ollama server is available at http://localhost:11434
def generate_text(prompt):
    url = 'http://localhost:11434/api/generate'
    headers = {
        'Content-Type': 'application/json'
    }
    data = {
        'model': 'llama3',
        'prompt': prompt,
        'stream': False
    }
    
    try:
        response = requests.post(url, headers=headers, data=json.dumps(data))
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f'Error during API call: {e}')
        return None

# Example usage for a simple prompt
result = generate_text('What is the capital of France?')
if result:
    print(result['response'])

# --- Advanced example for chat completion with history ---
def chat_completion(messages):
    url = 'http://localhost:11434/api/chat'
    headers = {
        'Content-Type': 'application/json'
    }
    data = {
        'model': 'llama3',
        'messages': messages,
        'stream': False
    }

    try:
        response = requests.post(url, headers=headers, data=json.dumps(data))
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f'Error during API call: {e}')
        return None

# Example chat history
chat_history = [
    {'role': 'user', 'content': 'What is the capital of France?'},
]

chat_result = chat_completion(chat_history)
if chat_result:
    print(chat_result['message']['content'])

You now have everything you need to start building and testing with AI locally. By using Ollama, you've unlocked a world of benefits: uncompromised privacy, zero running costs, and complete control over your environment. Whether you're running quick tests from the command line or building robust, automated suites with the REST API, you're now at the forefront of AI-powered quality assurance. The future of testing is here—and it's on your machine. Now go build something amazing!