6. Using an LLM with a Coding IDE

Turn your local LLM into a coding assistant by connecting it to your IDE. Learn how VS Code, Ollama, and coding models such as Qwen can work together to explain code, generate files, refactor projects, write tests, and assist with larger codebases.

A chatbot is useful for answering programming questions, but an LLM becomes considerably more useful when it is connected directly to your coding environment.

Instead of repeatedly copying code into a browser-based chatbot, an IDE-integrated model can work with the files you are actually editing. Depending on the tool and model, it can explain selected code, suggest changes, generate tests, search a codebase, modify files, and participate in agent-style coding workflows.

The basic architecture is:

┌──────────────────────┐ │ VS Code │ │ │ │ Editor + AI Tools │ └──────────┬───────────┘ │ │ Local API ▼ ┌──────────────────────┐ │ Ollama │ │ Inference Runtime │ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Qwen Coder │ │ Local Model │ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Your CPU / GPU / │ │ Unified Memory │ └──────────────────────┘ 

The important idea is that VS Code is the interface, Ollama is the local model server, and Qwen is the model doing the actual inference.

Why Use an LLM Inside an IDE?

An IDE gives the model something that a normal chatbot does not have: context about your actual project.

Depending on the integration, an AI coding assistant can work with:

• The file currently open in the editor.
• Selected sections of code.
• Other files in the project.
• Project documentation.
• Configuration files.
• Tests.
• Git repositories and version history.
• Terminal output and error messages.
• Tools exposed to the agent.

This changes the interaction from:

"What does this Python function do?" 

to:

"Find why this function is failing, identify the relevant code elsewhere in the project, and propose a fix." 

The second task requires the model to understand codebase context, not simply generate an isolated answer.

The Main IDE Options

There are several ways to integrate local AI into a coding environment. They provide overlapping functionality, but their architecture and workflows are different.

VS Code / VSCodium

VS Code has a huge extension ecosystem and now has built-in support for connecting external language models through its Bring Your Own Language Model (BYOK) system. Local models can be used in VS Code’s Chat experience without requiring a Copilot subscription or even a GitHub sign-in.

For Ollama specifically, current VS Code releases recommend the official Ollama extension rather than the older built-in Ollama provider, which is now deprecated.

VS Code also supports extensions such as Continue, which provides a more specialised local-AI coding workflow.

Continue

Continue is an open-source coding assistant that integrates directly into VS Code and JetBrains IDEs. It supports separate model roles such as chat, edit, apply, autocomplete, embedding, and reranking.

This separation is particularly useful with local models because the model that is ideal for autocomplete does not necessarily need to be the same model used for complex code reasoning.

Continue’s current documentation specifically supports Ollama and provides configurations for Qwen coding models, including Qwen2.5-Coder and newer Qwen coder models.

Cursor and Windsurf

AI-first editors such as Cursor and Windsurf build AI functionality more deeply into the editing experience.

They can provide highly integrated code generation, context gathering, chat, and agent-style editing. If your objective is specifically to learn how local models work, however, a conventional editor such as VS Code has the advantage of exposing the underlying components more clearly.

JetBrains IDEs

If you use PyCharm, IntelliJ IDEA, WebStorm, or another JetBrains IDE, local models can also be connected through extensions and provider integrations. Continue, for example, supports JetBrains environments as well as VS Code.

Neovim

For terminal-oriented developers, Neovim provides another route into local AI. Plugins such as avante.nvim and codecompanion.nvim can connect an editor workflow to local or OpenAI-compatible model servers.

This approach provides considerable flexibility, but generally requires more configuration than VS Code.

VS Code + Ollama: The Basic Setup

For learning local AI-assisted programming, VS Code combined with Ollama provides a straightforward architecture.

The setup consists of four components:

1. VS Code — your development environment.
2. Ollama — the local model server.
3. Qwen Coder — the coding model.
4. VS Code’s AI integration or Continue — the interface connecting your editor to the model.

Step 1: Install VS Code

Download and install Visual Studio Code for your operating system.

Once installed, open the project you want the model to work with using:

File → Open Folder 

It is important to open the project folder, rather than simply opening an individual source file. This gives your AI tooling a meaningful project context.

Step 2: Install Ollama

Install Ollama for your operating system and verify that it is running.

From a terminal, you can check the installation with:

ollama --version 

You can also check whether the Ollama service is available at its standard local endpoint:

http://localhost:11434 

Ollama normally exposes its local API on port 11434.

Step 3: Download a Coding Model

You now need a model specifically suited to programming.

For a lightweight local coding assistant, Qwen’s smaller coding models are useful because they can provide low-latency responses without requiring workstation-class hardware.

For example:

ollama run qwen2.5-coder:1.5b 

Or, where available for your Ollama installation, you can select a larger Qwen coding model appropriate to your hardware.

Continue’s current documentation specifically recommends Qwen2.5-Coder 1.5B for local autocomplete and documents Qwen3 Coder as an option for more substantial coding tasks.

Step 4: Test Qwen Through Ollama

Before involving VS Code, test the model directly.

ollama run qwen2.5-coder:1.5b 

Then give it a simple programming task:

Write a Python function that checks whether a string is a palindrome. 

If Ollama generates a sensible response, your model and runtime are working.

You can also test a larger model if your hardware permits it. The important principle is to establish that the model works independently before troubleshooting the IDE integration.

Step 5: Connect Ollama to VS Code

Modern VS Code provides a native model-management interface for connecting external language models.

Open the Command Palette:

Cmd/Ctrl + Shift + P 

Search for:

Chat: Manage Language Models 

Alternatively, use the model-management option from the language-model picker in the Chat interface.

VS Code’s current BYOK system allows local models to participate in the Chat experience without requiring a GitHub account or Copilot subscription.

For Ollama specifically, install the official Ollama VS Code extension. The current VS Code documentation identifies this as the recommended route because the older built-in Ollama provider is deprecated.

Step 6: Select Your Local Model

Once the Ollama integration is installed and your model is available, open the VS Code Chat interface.

You should be able to select the local model from the model picker.

The architecture is now:

VS Code Chat ↓ Ollama Extension / Provider ↓ localhost:11434 ↓ Qwen ↓ Your Hardware 

Your prompts are now being processed by the local model rather than a remote AI service.

Alternative: VS Code + Continue

If you want more specialised AI coding functionality, install the Continue extension from the VS Code Extensions marketplace.

Continue provides dedicated workflows for:

• Chat.
• Code editing.
• Applying generated changes.
• Autocomplete.
• Codebase context.
• File context.
• Model selection by task.

Its chat interface can include selected code and additional context using mechanisms such as @Files.

Step 7: Configure Continue with Ollama

Continue uses a configuration file to define the models and their roles.

A basic Ollama configuration has the following structure:

name: Local Qwen version: 0.0.1 schema: v1 models: - name: Qwen Coder provider: ollama model: qwen2.5-coder:7b 

The exact model identifier must match a model available in your Ollama installation.

Continue’s current configuration uses provider: ollama and can specify an apiBase when connecting to a non-local Ollama server.

Step 8: Configure a Fast Autocomplete Model

Autocomplete is a special case.

When you type:

def calculate_total( 

you don’t want to wait several seconds for a large reasoning model to decide what comes next.

You want a small model that can respond extremely quickly.

Continue’s current documentation uses Qwen2.5-Coder 1.5B as an example of a local autocomplete model:

models: - name: Qwen Autocomplete provider: ollama model: qwen2.5-coder:1.5b roles: - autocomplete 

Continue specifically notes that thinking-oriented models are generally unsuitable for autocomplete because they generate too slowly for an interaction that needs to feel immediate.

VS Code │ ┌──────────┴──────────┐ │ │ Autocomplete Chat │ │ ▼ ▼ Qwen Coder 1.5B Larger Qwen Coder │ │ └──────────┬──────────┘ ▼ Ollama 

Step 9: Give the Model Context

The quality of an IDE coding assistant depends heavily on the quality of the context you provide.

Instead of asking:

Fix this. 

give the model useful context:

Explain why this function is returning None. Look at the selected function and identify: 1. where the value is lost, 2. which caller expects the value, 3. the smallest safe fix, 4. a unit test that demonstrates the problem. 

In Continue, you can include files and other contextual information through its context mechanisms.

Codebase Context

One of the biggest advantages of an IDE integration is the ability to reason about more than the currently visible file.

For example, a useful request might be:

Find where UserService is called. Determine why the new return type breaks the existing API endpoint. Show me the affected files before suggesting a change. 

This requires the AI system to retrieve relevant portions of the project rather than blindly generating code based on one file.

Continue supports codebase-aware workflows and context selection, while VS Code’s own AI features have their own mechanisms for working with project context.

Step 10: Ask the Model to Explain Before Editing

When working with an unfamiliar codebase, don’t immediately give an AI agent permission to modify files.

A safer workflow is:

1. Explain the problem. ↓ 2. Identify relevant files. ↓ 3. Explain the proposed solution. ↓ 4. Review the plan. ↓ 5. Generate the changes. ↓ 6. Review the diff. ↓ 7. Run tests. ↓ 8. Commit the changes. 

This is particularly important with local models. A model running on your own computer can still make incorrect edits, introduce bugs, delete useful code, or misunderstand project architecture.

Using an LLM for Refactoring

LLMs are particularly useful for repetitive transformations.

For example:

Refactor this class to use dependency injection. Do not change the public API. Preserve existing behaviour. Update the unit tests. Show the proposed changes before applying them. 

This is a much better instruction than:

Make this code better. 

The more precisely you define the constraints, the easier it is for the model to produce a useful result.

Using an LLM to Write Tests

Testing is another particularly effective use of local coding models.

For example:

Analyse this function. Generate pytest tests covering: - normal input, - empty input, - invalid input, - boundary values, - expected exceptions. Do not modify the implementation. 

The model can then generate tests which you review and run locally.

This creates a useful feedback loop:

Human ↓ LLM generates code ↓ Test suite ↓ Actual result ↓ LLM analyses failure ↓ Human reviews fix 

Using an LLM for Debugging

One of the strongest IDE workflows is to give the model an actual error rather than asking it to guess what might be wrong.

For example:

The following pytest test fails: [paste error / terminal output] Here is the relevant function: [selected code] Find the most likely cause. Do not change the code yet. Explain the problem first. 

This approach gives the model concrete evidence rather than asking it to speculate.

Agent Mode

Modern coding assistants increasingly provide agent-style workflows.

Instead of simply returning code in a chat window, an agent may be able to:

• Inspect files.
• Search the project.
• Read documentation.
• Modify files.
• Run commands.
• Run tests.
• Inspect errors.
• Make additional changes.

This is considerably more powerful than ordinary autocomplete, but it also requires a stronger model and a more reliable tool-calling system.

Continue’s documentation explicitly identifies tool use as an important capability for Agent mode.

Choosing the Right Model Size

For local coding, model size should be chosen according to the task rather than simply selecting the largest model that fits.

For autocomplete, latency is especially important. A model that produces excellent suggestions but takes several seconds to respond will make the editor feel frustratingly slow.

Continue currently recommends Qwen2.5-Coder 1.5B as one local autocomplete option, while its model documentation also lists larger Qwen coder models for chat and more demanding tasks.

Context Length for Coding

Codebases can require considerably more context than a simple chatbot conversation.

A useful coding request might involve:

• A system prompt.
• The current file.
• Several related source files.
• Project documentation.
• Configuration.
• Previous conversation.
• Tool definitions.
• Terminal output.

This can quickly consume thousands or tens of thousands of tokens.

However, simply selecting the largest possible context window is not necessarily the best solution. Larger contexts can increase memory usage and processing time, particularly with local models.

Continue’s Ollama documentation specifically notes that its default context settings can sometimes require more system memory than other Ollama applications, and recommends reducing contextLength when memory becomes a problem.

Common Problems

http://localhost:11434 

VS Code’s Native Local AI vs Continue

These approaches solve slightly different problems.

There is an important current limitation in VS Code’s native BYOK approach: local BYOK models are available for Chat and related workflows, but standard inline code suggestions are not currently provided by the BYOK model itself. VS Code documents inline suggestions and embedding-dependent features as requiring Copilot or an extension-based completion provider.

A Recommended Learning Workflow

The Local AI Coding Stack

Once everything is configured, your development environment can be thought of as a series of layers:

┌─────────────────────────────┐ │ VS Code │ │ │ │ Editor / Chat / Extensions │ └──────────────┬──────────────┘ │ ▼ ┌─────────────────────────────┐ │ AI Integration │ │ │ │ Ollama Extension / Continue │ └──────────────┬──────────────┘ │ ▼ ┌─────────────────────────────┐ │ Ollama │ │ Local API Server │ └──────────────┬──────────────┘ │ ▼ ┌─────────────────────────────┐ │ Qwen Coder │ │ Quantised Model │ └──────────────┬──────────────┘ │ ▼ ┌─────────────────────────────┐ │ CPU / GPU / RAM │ │ or Unified Memory │ └─────────────────────────────┘ 

The Key Principle

An AI coding assistant is not simply a chatbot embedded inside an editor.

It is a system in which the IDE supplies context, the integration manages that context, Ollama provides the inference API, and the coding model generates or evaluates solutions.

The most effective workflow is therefore not:

Ask AI → Copy Code → Paste Code 

but:

Understand ↓ Provide Context ↓ Ask the Model ↓ Review the Proposal ↓ Apply the Change ↓ Run Tests ↓ Inspect the Result ↓ Iterate 

Once this workflow becomes familiar, you can move from simple code completion to increasingly sophisticated local AI development: code explanation → generation → testing → refactoring → debugging → codebase analysis → tool use → agentic software development.

The important lesson is to scale the model and the autonomy to the task. Use a small, fast model when latency matters, a larger coding model when deeper analysis is required, and give an agent only the tools and permissions it actually needs.