--- title: An entirely open-source AI code assistant inside your editor · Ollama Blog source: https://ollama.com/blog/continue-code-assistant created: 2024-10-12 description: Continue enables you to easily create your own coding assistant directly inside Visual Studio Code and JetBrains with open-source LLMs. tags: - clippings --- > This is a guest post from Ty Dunn, Co-founder of Continue, that covers how to set up, explore, and figure out the best way to use Continue and Ollama together. ![Continue and Ollama](https://ollama.com/public/blog/ollama-continue.png) [Continue](https://continue.dev/) enables you to easily create your own coding assistant directly inside Visual Studio Code and JetBrains with open-source LLMs. All this can run entirely on your own laptop or have Ollama deployed on a server to remotely power code completion and chat experiences based on your needs. To get set up, you’ll want to install - [Continue](https://docs.continue.dev/quickstart) for [VS Code](https://marketplace.visualstudio.com/items?itemName=Continue.continue) or [JetBrains](https://plugins.jetbrains.com/plugin/22707-continue) - [Ollama](https://github.com/ollama/ollama?tab=readme-ov-file#quickstart) for [macOS](https://ollama.com/download/mac), [Linux](https://ollama.com/download/linux), or [Windows](https://ollama.com/download/windows) Once you have them downloaded, **here’s what we recommend exploring:** ### Try out Mistral AI’s Codestral 22B model for autocomplete and chat As of the now, [Codestral](https://mistral.ai/news/codestral/) is our current favorite model capable of both autocomplete and chat. This model demonstrates how LLMs have improved for programming tasks. However, with 22B parameters and a [non-production license](https://mistral.ai/news/mistral-ai-non-production-license-mnpl/), it requires quite a bit of VRAM and can only be used for research and testing purposes, so it might not be the best fit for daily local usage. a. Download and run Codestral in your terminal by running ```undefined ollama run codestral ``` b. Click on the gear icon in the bottom right corner of Continue to open your `config.json` and add ```json { "models": [ { "title": "Codestral", "provider": "ollama", "model": "codestral" } ], "tabAutocompleteModel": { "title": "Codestral", "provider": "ollama", "model": "codestral" } } ``` ![VS Code settings to change config.json](https://ollama.com/public/blog/continue-settings-vscode.png) ### Use DeepSeek Coder 6.7B for autocomplete and Llama 3 8B for chat Depending on how much VRAM you have on your machine, you might be able to take advantage of Ollama’s ability to run multiple models and handle multiple concurrent requests by using [DeepSeek Coder 6.7B](https://deepseekcoder.github.io/) for autocomplete and [Llama 3 8B](https://ai.meta.com/blog/meta-llama-3/) for chat. If your machine can’t handle both at the same time, then try each of them and decide whether you prefer a local autocomplete or a local chat experience. You can then [use a remotely hosted or SaaS model](https://docs.continue.dev/setup/select-provider) for the other experience. a. Download and run DeepSeek Coder 6.7B in your terminal by running ```undefined ollama run deepseek-coder:6.7b-base ``` b. Download and run Llama 3 8B in another terminal window by running ```undefined ollama run llama3:8b ``` c. Click on the gear icon in the bottom right corner of Continue to open your `config.json` and add ```json { "models": [ { "title": "Llama 3 8B", "provider": "ollama", "model": "llama3:8b" } ], "tabAutocompleteModel": { "title": "DeepSeek Coder 6.7B", "provider": "ollama", "model": "deepseek-coder:6.7b-base" } } ``` ### Use `nomic-embed-text` embeddings with Ollama to power `@codebase` Continue comes with an [@codebase](https://docs.continue.dev/customization/context-providers#codebase-retrieval) context provider built-in, which lets you automatically retrieve the most relevant snippets from your codebase. Assuming you have a chat model set up already (e.g. Codestral, Llama 3), you can keep this entire experience local thanks to embeddings with Ollama and [LanceDB](https://blog.lancedb.com/lancedb-x-continue/). As of now, we recommend using `nomic-embed-text` embeddings. a. Download `nomic-embed-text` in your terminal by running ```undefined ollama pull nomic-embed-text ``` b. Click on the gear icon on the bottom right corner of Continue to open your `config.json` and add ```json { "embeddingsProvider": { "provider": "ollama", "model": "nomic-embed-text" } } ``` c. Depending on the size of your codebase, it might take some time to index and then you can ask it questions with important codebase sections automatically being found and used in the answer (e.g. “@codebase what is the default context length for Llama 3?”) ### Fine-tune StarCoder 2 on your development data and push it to the Ollama model library When you use Continue, you automatically generate data on how you build software. By default, this [development data](https://docs.continue.dev/development-data) is saved to `.continue/dev_data` on your local machine. When combined with the code that you ultimately commit, it can be used to improve the LLM that you or your team use (if you allow). For example, you can use accepted autocomplete suggestions from your team to fine-tune a model like StarCoder 2 to give you better suggestions. a. [Extract and load the “accepted tab suggestions” into Hugging Face Datasets](https://github.com/dlt-hub/continue-dlt-demo/blob/main/continue-hf-pipeline.py) b. [Use Hugging Face Supervised Fine-tuning Trainer to fine-tune StarCoder 2](https://colab.research.google.com/drive/1jjb14BDlEeGjRmeXnfm41gDBlTNvsscn) c. [Push the model to the Ollama model library for your team to use and measure how your acceptance rate changes](https://ollama.com/oakela/starcoder2_continue) ### Learn more about Ollama by using `@docs` to ask questions with the help of Continue Continue also comes with an [`@docs`](https://docs.continue.dev/customization/context-providers#documentation) context provider built-in, which lets you index and retrieve snippets from any documentation site. Assuming you have a chat model set up already (e.g. Codestral, Llama 3), you can keep this entire experience local by providing a link to the Ollama README on GitHub and asking questions to learn more with it as context. a. Type `@docs` in the chat sidebar, select “Add Docs”, copy and paste “[https://github.com/ollama/ollama”](https://github.com/ollama/ollama%22) into the URL field, and type “Ollama” into the title field b. It should quickly index the Ollama README and then you can ask it questions with important sections automatically being found and used in the answer (e.g. “@Ollama how do I run Llama 3?”) ## Join our Discord! Now that you have tried these different explorations, you should hopefully have a much better sense of what is the best way for you to use Continue and Ollama. If you ran into problems along the way or have questions, join the [Continue Discord](https://discord.com/invite/EfJEfdFnDQ) or the [Ollama Discord](https://discord.com/invite/ollama) to get some help and answers.