Evan X. Merz

In this post, I'm going to show you how you can generate and view a software bill of materials (sbom) for a node.js project.

Why generate a bill of materials?

With the increasing attacks on the software supply chain by malicious actors such as foreign states and crypto miners, it's important to understand exactly what packages are in your node project. It's also important to understand the vulnerabilities in each of those packages. The tools in this tutorial link out to Socket.dev, which analyzes npm packages to find potential vulnerabilities. You may also want to consult with the National Vulnerability Database to see if any of your packages have known security holes.

Steps

1. Install cyclonedx/bom. The CycloneDX SBOM Specification is a format for listing packages in a node library and any other software project. cyclonedx/bom is a tool for generating a sbom that conforms to the CycloneDX SBOM Specification.

npm install -g cyclonedx/bom

2. Use npx to run it. Navigate to the directory of your node project and run the following. This tells the package to generate a bill of materials by analyzing the node_modules folder in the current folder and save the output to bom.json.

npx @cyclonedx/bom . -o bom.json

3. Install viewbom. Viewbom is a simple npx tool I wrote that generates an html UI for navigating the bill of materials you just generated.

npm install -g viewbom

4. Run viewbom on the sbom you created.

npx viewbom bom.json bom.html

5. Open bom.html in a web browser. This should present you with a simple UI that shows you some basic statistics about your sbom and give you a way to search the packages in it.

It should look something like the following screenshot.

Why can't the developers move more quickly? Why does every project take so long? In this article I'm going to talk about what causes software development to slow down. This is based on my experience as a programmer and manager in Silicon Valley for over a decade, as well as conversations with other people in similar roles.

Is my team slow because...

First of all, I want to answer some questions that I've heard from managers.

1. Is my team slow because we use PHP? No.
2. Is my team slow because we use Java? No.
3. Is my team slow because we use INSERT PRODUCT HERE? Probably not.
4. Is my team slow because they don't understand web 3? No.
5. Is my team slow because of a bad manager? Maybe, but if you've had multiple managers in there who haven't moved faster then this is unlikely.
6. Is my team slow because they have or do not have college degrees? No.
7. Is my team slow because we don't use agile? Probably not.

So then why is my development team so slow?

Now that we've gotten that out of the way, we can drill into what actually causes teams to move slowly. None of the reasons I've listed here are particularly technical. I tend to think that teams move slowly due to non-technical reasons.

They're estimating poorly

The most common reason why people think developers are moving slowly is that the estimates are poor. For one reason or another, developers are not spending the time, or creating the artifacts necessary to generate accurate estimates.

I once joined a small team where nobody ever wrote anything down. They would attend planning meetings where they would hash out the requirements, they would make up an estimate on the spot, then they would immediately break to start writing code. The stakeholders wanted to know why tasks never seemed to come in within the estimated time frame.

In my experience, it's necessary to generate multiple levels of plans to generate an accurate software development project estimate. You must begin with use-cases, then work through multiple levels of plans to reach units of work that can be estimated.

Here's how I break down a project to make an accurate estimate:

1. Write all requirements down in plain language. This can be in the form of use cases, or user stories, or any top-level form that anyone can understand.
2. Talk through what an acceptable solution would look like with other developers.
3. Break that solution into individual tasks using a tool like JIRA.
4. Estimate those tasks.
5. Add a little extra room for unknowns, debugging, integration, and testing.
6. Add up the final estimate.

There are multiple ways to execute each of these steps, but that's the general flow for all successful software development estimation that I've seen in my career.

Technical debt

Another big reason why a team might be moving slowly is tech debt. Tech debt happens to every project when the company tries to move quickly. It happens because solutions that can be implemented as fast as possible are usually not scalable to future use cases that the developers can't anticipate.

I once worked on a web development team that needed to make the list pages faster on an e-commerce website. The list pages were slow because the server was delivering a massive payload containing every product on the list page then filtering and sorting was done client side. That's easy to fix right? Just serve the first page of products, then serve the next when the user scrolls down. But to do that, we must add server side sorting and filtering because the list will be different for different combinations of sorts and filters. But to do that, we must have products stored in some sort of database, and this team was only using a key-value store at the time. That's how tech debt can be problematic. In that scenario, it would take months to solve that problem in a scalable way.

Ultimately, we found a different solution. We delivered the tiniest possible payload for each product, which achieved a similar end goal while side-stepping the tech debt. But that tech debt was still there, and would still be a problem for many related projects.

There is no silver bullet for solving tech debt. In my career, the best way I've found to deal with tech debt is to reserve one developer each sprint to iteratively move the project closer to what the team agrees is a more scalable solution.

Interpersonal conflict

The single biggest thing that slows down development teams is interpersonal and interdepartmental conflict. Maybe a manager is at odds with a team member, or maybe two team members see their roles very differently. If two people working on one project have different views about how to execute that project it can be absolutely crushing.

I think that proactively avoiding and addressing conflict is the single thing that can instantly help teams move faster. The most staggeringly successful group of techniques I've discovered for doing so is called motivational interviewing.

Motivational interviewing is a way of communicating with peers, stakeholders, and direct reports that allows two people to agree to be on the same team. It's a way of building real rapport with people that allows you to build successful working relationships over time. Specifically, it's a way of focusing a conversation on talk that meaningfully moves both parties toward a mutually acceptable solution.

I strongly recommend Motivational Interviewing for Leadership by Wilcox, Kersh, and Jenkins. To say that that book changed my life is an understatement.

Unclear instructions

The final category of issue that slows down development teams is unclear instructions.

I once worked for a person who was very fond of sending short project descriptions over email and expecting fast results. One email might read something like "I want to be selling in Europe in one month", another might say "I want to be able to hold auctions for some of our products."

It IS possible to work with such minimal instructions. It falls upon the development team to outline what they think the stakeholder wants, develop plans based on that outline, then execute based on those plans.

The problem with working from minimal instructions is that no two people see the same problem in exactly the same way. Say you are developing that auction software. Many questions immediately arise. Who can bid on the auctions? Is the auction for many products or just one? Does the product have selectable options, or are they preset? Are bids visible to other customers? How do we charge the user for the item if the final bid exceeds what is typically put on a credit card?

If the stakeholder isn't able to answer those questions early in the process, then it may go significantly astray. If the development team doesn't understand the instructions, then they may waste days or weeks pursuing irrelevant or untimely work. If the stakeholder is going to give such vague instructions then they must address questions in a timely manner or risk tasting time.

In conclusion

So why is your development team so slow? I've laid out the four common causes that I've seen in my career.

1. Poor estimates
2. Tech debt
3. Interpersonal conflict
4. Unclear instructions

If you have something else to add to this list, then let me know on Twitter @EvanXMerz.

I recently attended the exhibit Sam Hyde Harris: Seeing the Unusual at Casa Romantica in San Clemente, California. The exhibit was curated by Maurine St. Gaudens and Joseph Marsman and brought together around 60 pieces by Sam Hyde Harris in just two rooms on the lovely Casa Romantica estate.

Sam Hyde Harris is mostly remembered by collectors today as an active member of the California Art Club who produced hundreds of beautiful landscapes of California in the early 20th century and taught numerous students who went on to great careers. He was also a member of a group called the California Impressionists.

The focus of this exhibit was introducing a modern audience to the massive volume of commercial work he produced. This includes numerous posters for companies such as Union Pacific and the Santa Fe Railroad, as well as advertising pieces for local businesses and theater companies.

Here's the Curator's Statement from Maurine St. Gaudens.

Sam Hyde Harris, Seeing the Unusual explores the diverse oeuvre of this noted twentieth century California artist. Although widely known for the fine art compositions, few people realize the extent of Harris' commercial advertising work. Harris' designs shaped the consciousness of early to mid-twentieth century consumers and travelers. I really was quite surprised that so little attention had been paid to Harris the commercial artist. This exhibition explores this complex aspect of the artist's career.

My personal association with Sam Hyde Harris actually began more than thirty years ago when I was contacted by Harris' widow, Marion Dodge Harris, to catalogue the artist's estate. In the process I discovered examples of work the artist had created for a who's who of clients, not only in California, but across the western United States and nationally. It's the commercial work that today represents an historical record of product lines and services that were a part of everyday life from the 1920s - 1950s.

On a national level, Harris had a long and highly creative relationship with the railroad industry, specifically the Santa Fe, Southern Pacific, and Union Pacific rail lines. His iconic Art Deco themed Southern Pacific's New Daylight poster has become one of the most recognizable images of this art form and one that has become highly praised by railroad and design enthusiasts alike.

Unfortunately, over the years, as is common with many commercial artists, their work has gone unaccredited, and Harris is no exception. Although, with new research and recent descoveries, this oversight is now being corrected, and Harris' commercial designs are being recognized by a new generation of historians.

Harris' commercial work wasn't the only thing in the exhibition, though. There were plenty of paintings of the subjects for which he was most famous, including boats in harbor, and the Chavez Ravine before it was developed.

I spent an hour poring over every piece in the exhibit. Many of the pieces came from the collection of Charles N. Mauch, including several massive paintings that became his most famous posters. I particularly enjoyed seeing the different stages of the Taxco Mission poster produced for Southern Pacific. Through a sketch, a painting, and a print of the final poster, visitors could see the complete evolution of one of his commercial pieces.

I only recently became a fan and collector of work by Sam Hyde Harris, and I was glad to have the opportunity to see so much of his work in one place. I was also glad that this less well-known California artist was being brought to new audiences in the 21st century.

Sam Hyde Harris: Seeing the Unusual ran from November 19, 2021 through February 27, 2022.

I recently purchased Emerging from the Shadows, Vol. I: A Survey of Women Artists Working in California, 1860-1960 and while reading it I discovered one particularly interesting artist named Viola M. Allen. Her facility with a palette knife seems almost miraculous, so of course I started searching the web for more of her work. Other than a few items available at auction, I haven't been able to find much information. So I wanted to share a couple quotes from the fabulous book by Maurine St. Gaudens.

Viola M. Allen was born on March 14, 1906, in Queens, New York, the daughter of Safarine D. Allen and Minnie (Eschman) Allen. According to her biographical artist's promotional card, she attended, in New York, the Pratt Institute and the National Academy of Design, where she studied under Charles Curran. Her card also indicates that she studied portrait painting with Moskowitz and Borgdonav and sculpture with Haffner and Monahan. A resident of Manhattan, New York, through the 1930s, by the latter part of the decade she had moved to Los Angeles, California; she remained a California resident until her death.

A study of Viola's paintings shows that she was a palette knife painter. Her ability to create realistic compositions by applying oil paint to a canvas, or board, by the use of a flexible painter's palette knife rather than a brush is found in most of her work; it is a difficult technique and one not widely practiced. Palette knives vary in length and width, and each one has a different tip, enabling the artist to achieve a different type of stroke, with the oil painting usually being applied very thickly on the canvas or board. During her career Viola had a commercial art studio in Malibu for many years where she did illustration and advertising art. In California, she exhibited with the California Art Club, 1955-1967.

This was all I could find out about her, and I'm happy to share it on the internet, and hopefully bring a little more attention to an artist who clearly had a control over the palette knife that few have ever achieved.

Of course I had to see if I could find one of her works at a reasonable price, and ebay came to my aid once again. I was able to find this beautiful small landscape listed for a song and now it hangs over my desk next to Sam Hyde Harris and Quincy Tahoma.

I don't think I've ever seen another artist wield a palette knife as fluently as she did, so I'll be on the lookout for more of her work. I hope that the internet can help preserve the legacy of a great artist who clearly deserves a re-evaluation.

One question I hear a lot lately is when a company should stick with Heroku or switch to something else. In this article I'm going to lay out the pros and cons for Heroku, and compare it with the typical alternative, AWS.

Three reasons to stick with Heroku

Don't fall victim to thinking that "the grass is always greener on the other side." That hot new technology on AWS or Azure may look cool now, but Heroku offers a lot of great features that should fit the bill for many growing companies.

Heroku supports easy scaling with sticky sessions

Horizontally scaling any web application is hard. In a traditional web app, you must optimize your code so that it runs in parallel. You must deal with race conditions that arise when multiple servers are trying to interact with a shared resource such as a cache or a database. You must find a way to balance load across multiple servers while sharing state across all instances.

Session data is visitor-specific data that is stored on the server. If a visitors's session is stored on one server, but their request is routed to another server, then that server won't know about anything they've done in the current session. It may not know if they're logged in or not. It may not have the browsing filters that they've configured.

Session affinity, also known as sticky sessions, is one solution to the problem of sharing session across servers. With session affinity enabled, all of a visitor's requests will always be routed to the same instance. There are some drawbacks to session affinity, but the benefit of being able to scale horizontally before having to tackle some parallelization problems may outweigh the drawbacks in your use case.

Many services offer session affinity, but none are as easy to set up as Heroku. With literally two clicks you can enable session affinity and start scaling out. Don't let hype distract you from an approach that may reap benefits for your web property.

Heroku provides zero down time deploys and upgrades

One of the best features of Heroku is that their technology and support teams handle deploys and upgrades. They make it so that your dev team doesn't have to worry about keeping the servers up during deploys or running migrations during upgrades.

The preboot feature allows you to keep the old version of your app running during a deploy. This means that users will only ever be routed to a server that has booted up and is running your app, and that means that you aren't turning away customers in that 30 second window where the new version of your app is loading.

Heroku also supports seamless upgrades for the most popular add-ons, such as REDIS. When I switched my company from Heroku REDIS to AWS REDIS we were surprised that our site went down a few weeks after the switch. AWS may force upgrade your technology without providing a way to seamlessly switch to the new version. So AWS forces you to track each upcoming patch and ensure that your team is ready for the switch.

Heroku is cheaper than the alternative because anyone can use it

Heroku seems very expensive when the bills come due, but in my experience it's cheaper than the alternative. Heroku is so easy to use that anyone can use it. With a few clicks or commands a backend developer can enable sticky sessions. With a few clicks or commands they can enable preboot. With a few clicks or commands they can add REDIS or PostGreSQL or any of the many add-ons provided by Heroku.

To use all those different products on a less managed product, such as AWS or Azure, you must retain a dedicated DevOps specialist. These people have very specialized skills and are not cheap. In my experience, using Heroku saves the cost of around one expensive employee. So as long as your Heroku bill is less than the cost of one employee, it's probably the more affordable option.

Three reasons to switch

There are many good reasons to stick with Heroku, but it certainly has limits. Here are the reasons why I've moved services from Heroku to somewhere else in the past.

Heroku is dangerous because anyone can use it

When you're on Heroku, you may not need to hire dedicated staff to manage your web infrastructure. This is a significant cost savings, but it means that the DevOps tasks are going to be offloaded on your web programmers. So you must ensure that you hire the skills on your team to understand Heroku. Heroku may not be as complex as AWS, but it still requires a foundational understanding of how the web works, linux, and logging. If your programmers are exploring the features in Heroku without the requisite experience or training then they may make mistakes that harm your business.

Heroku offers fewer options for international support

Heroku lacks flexible support for internationalized websites. As it says in the regions documentation, each "Private Space exists in a single region, and all applications in the Private Space run in that region.". This may sound confusing, but it ultimately means that each project in Heroku can only run in one region. If you want to support another region, then it must be in a separate project and hosted at a separate domain or subdomain. So if you want to internationalize your website using subdirectories, which is advantageous because it inherits the existing search reputation of your domain, then you can't do that with geographically distributed servers on Heroku.

Heroku offers fewer vertical scaling options

Heroku dynos come in six different flavors as of this writing. If you need anything outside of those six options then you are out of luck. The beefiest dyno is the performance-l machine, which offers 14Gb of RAM. If you need more than that, then you need to switch to another platform. What if you're using very little memory, but you want to use many CPU cores? The only option is to pay for the most expensive dynos. This lack of flexibility means that if your web service is a pretty standard website or API, then Heroku probably won't serve your needs very well.

How to decide whether to stick with Heroku or to switch

In this article I've listed some reasons to stick with Heroku and some reasons to switch, but the final decision is largely dependent on your use case. If you are making a pretty standard website or API, then Heroku is probably fine even when horizontally scaling. There are three main scenarios where you should strongly consider switching to a more complex cloud hosting service.

1. Your service need more flexible options for internationalization
2. Your service doesn't match the requirements common to most web apps and APIs
3. Your service is scaling exponentially