Evan X. Merz

This post is my attempt to share a simple "Hello, World" application using NodeJS and Ollama. It's a simple web API that allows users to chat with an LLM.

How to build a simple LLM chat API using NodeJS and Ollama

I'm going to show you the simplest way to do this, but you DO need to have some understanding of web programming in order to make this happen. I'm assuming that you have nodejs and npm installed on your computer. I'm assuming that you know how to run console commands.

1. Install the Ollama app

If you run your LLM using the Ollama app, then it will automatically run when you boot into Windows. This is convenient because you don't need to remember to start a second process or run a virtual machine.

Here's the link to download and install the Ollama app for Windows.

NOTE: This app requires a restart, which was not indicated in the installation instructions when I installed it.

you will also need to open the Ollama app, go to Settings, and enable "Expose Ollama to the network". This option enables the local Ollama API that will be used by this example.

Finally, you will need to install the model that you intend to use. In the code below, you will see that I'm using llama3.2:1b, the 1 billion parameter open source model released by Meta. I like this model because it runs really fast on my local machine and it gives reasonably good output most of the time for most tasks.

2. Create a new nodejs application and install dependencies

Create a folder for your nodejs application, then run npm init and give the project any details you like. We will use app.js as our entry point, but since we aren't sharing this project, that isn't actually important.

npm init

The only dependencies for this example are express, ollama, and body-parser. Install them with the following command.

npm i express ollama body-parser

Then you need to set up a command to run the project. Modify the scripts section of package.json to tell npm to run the node server.

"scripts": { "dev": "node app.js" },

After writing our server script, we can then use the following command to run the API.

npm run dev

3. Write the code

Type this code into app.js in the root folder.

DON'T COPY PASTE THIS. If you copy paste then you won't learn anything. Type it out line by line.

/**
 * This project is a simple demonstration of how to build a simple
 * node/express API on top of the ollama API.
 */
const express = require('express');
const ollamaLibrary = require('ollama');
const bodyParser = require('body-parser');
const port = 8181;
const DEFAULT_MODEL = "llama3.2:1b";

// instantiate the express server
const app = express();

// tell express to use the body-parser package to extract json from a post
app.use(bodyParser.json());

const ollama = new ollamaLibrary.Ollama({
  url: "http://localhost:11434"
});

/**
 * Set up a single endpoint for chatting with an llm.
 */ 
app.post('/chat', async (req, res) => {
  let params = req.body;
  let prompt = params?.prompt;
  if(prompt != null && prompt.length > 0) {
    let model = params?.model ?? DEFAULT_MODEL;
    const response = await ollama.chat({
      model: model,
      messages: [{ role: 'user', content: prompt }],
      stream: false
    });
    console.log(response.message.content);
    res.status(200);
    res.json({ response: response.message.content });
  } else {
    res.status(400).send("Invalid prompt.");
  }
});

/**
 * Start the server, and listen for requests.
 */ 
app.listen(port, () => {
  console.log(`Server started and listening on port ${port}...`);
});

4. Test using Yaak or Postman

To test that the system is working, you can use an API testing tool like Yaak or Postman. Just send a post request to http://localhost:8181/chat with the following body json.

{
  "prompt": "Why is the sky blue?"
}

It should look something like this in Yaak.

Continuing to build a real LLM API

If you were going to extend this into being a real API, then you would want to add authentication and authorization, CORS support, and more. The hardest part about building an API like this is not necessarily the code, but the infrastructure to support it in production.

Stop using expensive LLMs in VS Code. You bought a nice computer for writing code and playing videogames. It can run a large language model locally, and you can get the benefit of coding with AI without paying the price.

Also, running a model locally means that the code never leaves your computer. This can be a requirement if you're working on a highly sensitive codebase, or just want to make sure that you aren't giving away company secrets.

How to use a free LLM in VS Code

There are several ways that you could set this up. In this post I'm going to show you the easiest way to set it up, but it's not necessarily the most stable way to set it up, and some users may find that this doesn't work out of the box. If you want to run your LLM using Docker, then check out this prior post to see how.

1. Install the Ollama app

If you run your LLM using the Ollama app, then it will automatically run when you boot into Windows. This is convenient because you don't need to remember to start a second process or run a virtual machine.

Here's the link to download and install the Ollama app for Windows.

NOTE: This app requires a restart, which was not indicated in the installation instructions when I installed it.

2. Install the Ollama VS Code extension

Next you need to install the Ollama VS Code extension. The things that are underlined in red in this image are what you need to click.

Click on the extensions icon in the left sidebar. It's the four boxes where one is crooked. Then type Ollama into the search bar, select the first result, and click the small blue install button.

3. Run the Ollama VS Code harness

I'm not sure I'm getting the terminology correct for this step because the documentation is totally out of sync across all sources, and this is rapidly evolving. BUT you will need to run the following script to connect Ollama to VS Code.

ollama launch vscode

This will ask you to select a model, which it will then install. It doesn't seem to support every model, so I selected gemma4:26b for my model.

This requires a restart of VS Code after the command completes.

Hardware Note: Large models require significantly more memory. The number after the colon in the model name indicates the number of parameters for that model, which correlates with the amount of memory that the model will use. Start with small models like tinyllama, then try models with 1b (1 billion) parameters if your computer is struggling with the larger models.

4. Open the VS Code chat interface and begin prompting

Click "View" > "Chat". That will open the chat interface. Then you can immediately begin prompting. Yes, it will be much slower than the commercial models hosted online, but it will be free.

You can keep paying exorbitant fees to cloud LLM providers. I can't stop you. But, if you have a reasonably good computer, it's quite possible to run the models locally without paying a dime.

Will the model you run locally be as good as the best models offered by expensive cloud services such as Claude Code? No. Will they be sufficient to your purposes? Perhaps.

They also have the additional benefit of total privacy. You can disconnect your machine from the internet and ensure that everything you say to your LLM stays local on your computer.

How to run a model on your local machine using ollama

There are several ways to run a model locally. You could just use the Ollama app and pick a model from there, however, the app appears to be non-functional for me. I select a model and nothing happens. I send a chat and nothing happens.

So for now, you might need to do a little more than just download the app.

Here's how to run a model on your local machine using the Ollama Docker image.

1. Install docker.

Docker is a system for distributing software in virtual bundles called containers. It allows users to create and distribute an "image" that is essentially a virtual computer in one file. So one user can put an operating system and a bunch of software into a Docker image, then users who want to run that software can do so by running that image.

The easiest way to use Docker on Windows is to install Docker Desktop.

2. Pull the Ollama docker image.

Next, you will need to open a new command line window (command line = terminal = console). I use Windows PowerShell, but good old fashioned cmd should work just as well. You may need to install the WSL (Windows Subsystem for Linux). I can't recall if I needed to install that for this or not.

You will also need to have Docker running, which you can do simply by starting the Docker Desktop app.

Use the following command to pull the Ollama Docker image.

docker pull ollama/ollama

3. Run the Ollama Docker image.

Then you need to run the Ollama Docker image. This image will allow you to download and run models. You can do this in Docker Desktop using the UI, but I usually just do it in the console.

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

After you run this command once, the same container can be restarted using Docker Desktop.

4. Download and run the model you want.

The you need to select the model you want to run and run it. The smallest model is tinyllama.

docker exec -it ollama ollama run tinyllama

But the results from that model are pretty bad. I prefer the 1 billion parameter model from Meta, llama3.2:1b.

docker exec -it ollama ollama run llama3.2:1b

That should leave you with a command prompt where you can pose questions to your LLM.

Is The Odyssey, by Christopher Nolan, the best movie ever made? I'm not sure, but I think it belongs in that conversation.

It succeeds on every level. The writing is impeccable. The story is satisfying and rich (it's almost like it has been refined for thousands of years). The world feels real and lived in. All the characters, even small ones like Helen of Troy, have meaningful character arcs. The themes are deep and reveal powerful truths about human nature.

It succeeds in every possible genre you could put it into. As an action, adventure, or war movie, it is possibly the best ever made. It then somehow succeeds simultaneously as both a horror movie and a love story. It is successful as a monster movie and a coming of age tale.

It succeeds in every craft involved in modern cinema. The acting is inspired. The cinematography is literally breathtaking. The sound effects and mixing will punch you in the gut. Repeatedly. The special effects are top notch.

I've seen some complaints about the costumes. They seem to revolve around "historical accuracy" and Agamemnon's costume looking plasticky. As a viewer, you need to keep in mind that this isn't a tale from history, it's historical fiction. It's a fantasy movie, with magic, gods, and monsters. It has more in common with Game of Thrones than with Gettysburg. Yes, this is visible in the costumes.

I also see some complaints about the acting in spots, but I can't relate to those complaints at all. The acting seems entirely appropriate within the context of the film, and I think even includes some best-of-career performances by the top cast.

I just saw it two nights in a row, and I'd watch it a third time if a friend asked.

Ignore the trailers; they show nothing that matters to the movie. Ignore the fake controversies.

Just go see it, and prepare for a ride.

I've been employed in tech for over twenty years now, and for the fifth time I find myself in the position of seeking a new job. In my career I've had a lot of titles. I've been a "Programmer", a "Web Developer", a "Lead Backend Engineer", a "Staff Software Engineer", an "Engineering Manager", and a "Director of Software Development". In all of these roles, I've either been writing a lot of code, or I've been responsible for a large codebase. When I interviewed for all of these roles, I had to go through several rounds of coding interviews. So when preparing to interview, I spent a lot of time on LeetCode.

As any programmer can tell you, LeetCode is a platform for practicing the type of problems that frequently appear on coding interviews. In fact, you can practice for almost any question you will get on a coding interview on LeetCode. If LeetCode doesn't have the exact problem in its database, then it will have an analogous problem that can be solved in the same way.

As I enter another round of interviews in 2026, I find myself wondering if LeetCode remains as relevant today as it was in the past. Are these types of coding questions still relevant to programming when AI is going to be writing so much of the code? Does it still matter if I know all the different search algorithms? Does it matter if I have an encyclopedic knowledge of data structures?

The changing expectations for software developers

Whether you like it or not, the expectations placed on a software developer are changing. The people in those roles today are not doing the same thing that they were ten years ago, or even five years ago. What they will be doing five years from now is even less clear, but based on my experience at both startups and in big tech, I can see a few trends emerging.

More speed. More volume. More code.

By incorporating new AI models into developer workflows, developers are expected to produce more code faster. This means both writing more code, AND reviewing more code.

In the four years at my most recent job, the expectations for writing code more than doubled. When I started there, developers were expected to submit around two pull requests (aka diffs) per week. When I left, developers were expected to submit more than one pull request (diff) per day.

A similar thing occurred with code reviews. When I started there in 2022, we were expected to merge (aka land) our pull requests in a timely mannger. When I left in 2026, we were expected to review hundreds of pull requests (diffs) per quarter.

Participation in all aspects of software development

Developers are also expected to be able to spend more time doing things other than writing code. Because the AI is freeing up time we would have spent understanding a codebase, we are expected to invest that time into other aspects of development, such as planning, design, and analysis of the resulting work.

It's expected that most software developers today can contribute to all stages of software development. This means understanding how to use visual design tools such as Figma and Adobe products, but it also means understanding how to measure the success of your work using analytics and basic statistical tools such as A/B testing.

During my time in big tech, it was expected that I could quantitatively show the revenue impact of my work by setting up and running experiments. I had to discover the project, push the design process, write the code, run the experiment, then present the results to my peers and defend the value of the work.

Not all companies may be as demanding as big tech, but all programmers are expected to have some skills beyond just the ability to write code in 2026.

How does LeetCode help developers in 2026?

Amidst all this change, it's reasonable to ask whether a website that helps developers break down problems quickly and regurgitate known solutions is still valuable. I think that LeetCode is still valuable because it helps with two big problems facing software developers today: getting the job, and maintaining a high bar for software quality.

LeetCode helps you get the job

The fact remains that coding interviews are a prerequisite to doing the the job. You will never get the opportunity to write code using the newest AI models if you can't pass the coding interviews. LeetCode is still one of the best ways to prepare for coding interviews that aren't going away any time soon.

LeetCode helps you apply and recognize data structures and algorithms in real world situations

Despite all this change, the core problem of software development is still the same thing that it has always been: maintaining complexity.

The hard part of software development has never been the writing of the code. The hard part of software development has always been writing the code in such a way that future development and maintenance of the product is still feasible.

The challenge of integrating AI into software development is that it allows developers to write code much faster, but it also allows them to make mistakes much faster. This means that being able to recognize non-scalable patterns, inefficient data structures, and slow algorithms is critically important.

After all, when you are expected to review hundreds of pull requests per quarter, you have to learn to spot the things that will harm software development in the future.

By offering many different toy problems, LeetCode trains developers to see the things that might slow them down in the future. LeetCode trains developers to have algorithmic fluency. It trains them to see the similar algorithms and data structures that underlie problems that may appear quite different on the surface.

So I'm going to keep on training on LeetCode.