How to build a full stack LLM chat service with Ollama and JavaScript
This post is my attempt to share a full stack "Hello, World" application using JavaScript, NodeJS and Ollama. It's a simple website that allows users to chat with an LLM.
This example doesn't require you to sign up with cloud service providers or run anything outside of your own computer. The model, the api, and the frontend JavaScript all run on your local machine.
In a production situation, these tasks would be split out onto multiple computers with a lot of RAM and VRAM. The API would run on one web server, the model on another, and the frontend on a client computer. So unless you have a pretty beefy machine, you should expect the result to be slow.
How to build a full stack LLM service with Ollama and JavaScript
There are many ways to do this, but you DO need to have some understanding of web programming in order to make this happen. I'm assuming that you have nodejs and npm installed on your computer. I'm assuming that you know how to run console commands.
1. Get the API running
In a previous post, I showed you how to put up a simple web API to chat with an LLM. We're going to start this full stack project with that API as a base. So go execute that tutorial, then come back here when you have a functioning API. Make sure to validate your API by running it, and sending a web request to it using a client like Yaak.
2. Create the JavaScript frontend
Now we need a script to actually access the API. We will have our same project serve a static html file with a basic chat interface.
This html file will include a text box for entering a prompt, a submit button, a container for the response, and a title. I'm leaving it deliberately simple in order to demonstrate the basic interconnection between the different services involved.
Type this into a file called index.html in the root of your project.
<html>
<head>
<script>
async function handleSubmit() {
let prompt = document.getElementById("prompt").value;
if(prompt != null && prompt.length > 0) {
// hit our local chat API
let response = await fetch("http://localhost:8181/chat", {
method: "POST",
headers: {
Accept: "*/*",
"Content-Type": "application/json",
},
body: JSON.stringify({
prompt: prompt,
})
});
// interpret the response
if(response.body != null) {
const reader = response.body.getReader();
let text = "";
let chunk = await reader.read();
// this should be using the done parameter to chunk out the response, but there is some implementation issue that I'm just going to ignore...
let newChatChunk = new TextDecoder().decode(chunk.value);
text += newChatChunk;
let responseAsObject = JSON.parse(text);
document.getElementById('response-container').innerText = responseAsObject.response;
}
}
}
</script>
</head>
<body>
<h1>My amazing LLM chat service</h1>
<div id="response-container"></div>
<div id="input-container">
<input type="text" id="prompt" value="Why is the sky blue?" />
<button onClick="handleSubmit()">Submit</button>
</div>
</body>
</html>
3. Modify the Node/Express server to serve the HTML file
Next we need to modify our node/express server to have a frontend. We will add a route that serves the html file we just created.
Add the following code to app.js below the API endpoint.
app.get('/', (req, res) => {
res.sendFile('./index.html', { root: __dirname });
});
4. Run the app
Finally, you just need to run the web server. Remember that you need to restart Node/Express every time you make a change. Use the following command to run the server if you followed my instructions in the prior post.
npm run dev
Now the chat interface should be visible and functional at http://localhost:8181/.
Extending your basic chat service
I hope this template provides a good starting point for you. When you extend this, you will need to provide all the basic stuff that is important to a website, https support, CORS, authentication, and more.
The larger issue is that the LLM is running on the same server/computer as the website and API. The first problem you need to solve when building a service like this is how to run the model in a scalable way, and how to connect it to the user facing part of your website.
How to build a simple LLM chat api using NodeJS and Ollama
This post is my attempt to share a simple "Hello, World" application using NodeJS and Ollama. It's a simple web API that allows users to chat with an LLM.
How to build a simple LLM chat API using NodeJS and Ollama
I'm going to show you the simplest way to do this, but you DO need to have some understanding of web programming in order to make this happen. I'm assuming that you have nodejs and npm installed on your computer. I'm assuming that you know how to run console commands.
1. Install the Ollama app
If you run your LLM using the Ollama app, then it will automatically run when you boot into Windows. This is convenient because you don't need to remember to start a second process or run a virtual machine.
Here's the link to download and install the Ollama app for Windows.
NOTE: This app requires a restart, which was not indicated in the installation instructions when I installed it.
you will also need to open the Ollama app, go to Settings, and enable "Expose Ollama to the network". This option enables the local Ollama API that will be used by this example.
Finally, you will need to install the model that you intend to use. In the code below, you will see that I'm using llama3.2:1b, the 1 billion parameter open source model released by Meta. I like this model because it runs really fast on my local machine and it gives reasonably good output most of the time for most tasks.
2. Create a new nodejs application and install dependencies
Create a folder for your nodejs application, then run npm init and give the project any details you like. We will use app.js as our entry point, but since we aren't sharing this project, that isn't actually important.
npm init
The only dependencies for this example are express, ollama, and body-parser. Install them with the following command.
npm i express ollama body-parser
Then you need to set up a command to run the project. Modify the scripts section of package.json to tell npm to run the node server.
"scripts": { "dev": "node app.js" },
After writing our server script, we can then use the following command to run the API.
npm run dev
3. Write the code
Type this code into app.js in the root folder.
DON'T COPY PASTE THIS. If you copy paste then you won't learn anything. Type it out line by line.
/**
* This project is a simple demonstration of how to build a simple
* node/express API on top of the ollama API.
*/
const express = require('express');
const ollamaLibrary = require('ollama');
const bodyParser = require('body-parser');
const port = 8181;
const DEFAULT_MODEL = "llama3.2:1b";
// instantiate the express server
const app = express();
// tell express to use the body-parser package to extract json from a post
app.use(bodyParser.json());
const ollama = new ollamaLibrary.Ollama({
url: "http://localhost:11434"
});
/**
* Set up a single endpoint for chatting with an llm.
*/
app.post('/chat', async (req, res) => {
let params = req.body;
let prompt = params?.prompt;
if(prompt != null && prompt.length > 0) {
let model = params?.model ?? DEFAULT_MODEL;
const response = await ollama.chat({
model: model,
messages: [{ role: 'user', content: prompt }],
stream: false
});
console.log(response.message.content);
res.status(200);
res.json({ response: response.message.content });
} else {
res.status(400).send("Invalid prompt.");
}
});
/**
* Start the server, and listen for requests.
*/
app.listen(port, () => {
console.log(`Server started and listening on port ${port}...`);
});
4. Test using Yaak or Postman
To test that the system is working, you can use an API testing tool like Yaak or Postman. Just send a post request to http://localhost:8181/chat with the following body json.
{
"prompt": "Why is the sky blue?"
}
It should look something like this in Yaak.
Continuing to build a real LLM API
If you were going to extend this into being a real API, then you would want to add authentication and authorization, CORS support, and more. The hardest part about building an API like this is not necessarily the code, but the infrastructure to support it in production.
How to use a free LLM in VS Code
Stop using expensive LLMs in VS Code. You bought a nice computer for writing code and playing videogames. It can run a large language model locally, and you can get the benefit of coding with AI without paying the price.
Also, running a model locally means that the code never leaves your computer. This can be a requirement if you're working on a highly sensitive codebase, or just want to make sure that you aren't giving away company secrets.
How to use a free LLM in VS Code
There are several ways that you could set this up. In this post I'm going to show you the easiest way to set it up, but it's not necessarily the most stable way to set it up, and some users may find that this doesn't work out of the box. If you want to run your LLM using Docker, then check out this prior post to see how.
1. Install the Ollama app
If you run your LLM using the Ollama app, then it will automatically run when you boot into Windows. This is convenient because you don't need to remember to start a second process or run a virtual machine.
Here's the link to download and install the Ollama app for Windows.
NOTE: This app requires a restart, which was not indicated in the installation instructions when I installed it.
2. Install the Ollama VS Code extension
Next you need to install the Ollama VS Code extension. The things that are underlined in red in this image are what you need to click.
Click on the extensions icon in the left sidebar. It's the four boxes where one is crooked. Then type Ollama into the search bar, select the first result, and click the small blue install button.
3. Run the Ollama VS Code harness
I'm not sure I'm getting the terminology correct for this step because the documentation is totally out of sync across all sources, and this is rapidly evolving. BUT you will need to run the following script to connect Ollama to VS Code.
ollama launch vscode
This will ask you to select a model, which it will then install. It doesn't seem to support every model, so I selected gemma4:26b for my model.
This requires a restart of VS Code after the command completes.
Hardware Note: Large models require significantly more memory. The number after the colon in the model name indicates the number of parameters for that model, which correlates with the amount of memory that the model will use. Start with small models like tinyllama, then try models with 1b (1 billion) parameters if your computer is struggling with the larger models.
4. Open the VS Code chat interface and begin prompting
Click "View" > "Chat". That will open the chat interface. Then you can immediately begin prompting. Yes, it will be much slower than the commercial models hosted online, but it will be free.
How to run an LLM on a Windows computer using Ollama
You can keep paying exorbitant fees to cloud LLM providers. I can't stop you. But, if you have a reasonably good computer, it's quite possible to run the models locally without paying a dime.
Will the model you run locally be as good as the best models offered by expensive cloud services such as Claude Code? No. Will they be sufficient to your purposes? Perhaps.
They also have the additional benefit of total privacy. You can disconnect your machine from the internet and ensure that everything you say to your LLM stays local on your computer.
How to run a model on your local machine using ollama
There are several ways to run a model locally. You could just use the Ollama app and pick a model from there, however, the app appears to be non-functional for me. I select a model and nothing happens. I send a chat and nothing happens.
So for now, you might need to do a little more than just download the app.
Here's how to run a model on your local machine using the Ollama Docker image.
1. Install docker.
Docker is a system for distributing software in virtual bundles called containers. It allows users to create and distribute an "image" that is essentially a virtual computer in one file. So one user can put an operating system and a bunch of software into a Docker image, then users who want to run that software can do so by running that image.
The easiest way to use Docker on Windows is to install Docker Desktop.
2. Pull the Ollama docker image.
Next, you will need to open a new command line window (command line = terminal = console). I use Windows PowerShell, but good old fashioned cmd should work just as well. You may need to install the WSL (Windows Subsystem for Linux). I can't recall if I needed to install that for this or not.
You will also need to have Docker running, which you can do simply by starting the Docker Desktop app.
Use the following command to pull the Ollama Docker image.
docker pull ollama/ollama
3. Run the Ollama Docker image.
Then you need to run the Ollama Docker image. This image will allow you to download and run models. You can do this in Docker Desktop using the UI, but I usually just do it in the console.
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
After you run this command once, the same container can be restarted using Docker Desktop.
4. Download and run the model you want.
The you need to select the model you want to run and run it. The smallest model is tinyllama.
docker exec -it ollama ollama run tinyllama
But the results from that model are pretty bad. I prefer the 1 billion parameter model from Meta, llama3.2:1b.
docker exec -it ollama ollama run llama3.2:1b
That should leave you with a command prompt where you can pose questions to your LLM.
Is The Odyssey the best movie ever made?
Is The Odyssey, by Christopher Nolan, the best movie ever made? I'm not sure, but I think it belongs in that conversation.
It succeeds on every level. The writing is impeccable. The story is satisfying and rich (it's almost like it has been refined for thousands of years). The world feels real and lived in. All the characters, even small ones like Helen of Troy, have meaningful character arcs. The themes are deep and reveal powerful truths about human nature.
It succeeds in every possible genre you could put it into. As an action, adventure, or war movie, it is possibly the best ever made. It then somehow succeeds simultaneously as both a horror movie and a love story. It is successful as a monster movie and a coming of age tale.
It succeeds in every craft involved in modern cinema. The acting is inspired. The cinematography is literally breathtaking. The sound effects and mixing will punch you in the gut. Repeatedly. The special effects are top notch.
I've seen some complaints about the costumes. They seem to revolve around "historical accuracy" and Agamemnon's costume looking plasticky. As a viewer, you need to keep in mind that this isn't a tale from history, it's historical fiction. It's a fantasy movie, with magic, gods, and monsters. It has more in common with Game of Thrones than with Gettysburg. Yes, this is visible in the costumes.
I also see some complaints about the acting in spots, but I can't relate to those complaints at all. The acting seems entirely appropriate within the context of the film, and I think even includes some best-of-career performances by the top cast.
I just saw it two nights in a row, and I'd watch it a third time if a friend asked.
Ignore the trailers; they show nothing that matters to the movie. Ignore the fake controversies.
Just go see it, and prepare for a ride.