Web History

Recientemente escuché esta serie de podcasts sobre la historia de la web y me encantaron. Creo que cualquiera que trabaje construyendo la web debería conocer un poco sobre su historia y esta serie me parece ideal para eso.

Muy recomendado.


Categorized as Default

You Don’t Need NPM Scripts

If you have done any Node.js development, you’ve likely used NPM, and you should know that on your package.json file, you can add simple scripts that can be executed using npm run <name of script>. At first, this might seem convenient, and since most projects use it, why not use it.

In my experience, NPM scripts have a few problems.

  • They are encoded as JSON strings that don’t have syntax highlighting or linting by default on most editors.
  • Strings in JSON must be double-quoted, which produces the need to escape double quotes inside your scripts.
  • Longer scripts make your package.json difficult to read and understand.
  • On larger projects, you might need many scripts which can become unmanageable quickly. 
  • Scripts are all on a single namespace forcing you to invent naming conventions like build:css, build:js, build:production:js, etc.

The only good reason to use NPM scripts is to hook into life cycle events like postinstall, prestart, and friends. If you need something more than mocha src/**/*.test.js, you will be better off avoiding NPM scripts.

My suggestion is to create a top-level folder named scripts and organize it however you like. Inside the scripts folder, you will have executable files without file extensions, and each file must begin with the correct shebang line. 

Here’s an example from one of my projects. This file is located at scripts/build, which can be invoked by running ./scripts/build on the root of the project.

#!/usr/bin/env bash
# Build project for production
set -euf -o pipefail

echo "-> Building App..."

# Remove old files if they exist
rm -rf ./build

# Run postcss build

# Copy images to build

# Run server TypeScript build
echo "-> Compiling server-side TypeScript..."
npx tsc

# Run client side JS build and minify

# Copy other files to build dir
echo "-> Copiying *.hbs and *.json files..."
npx copyfiles --up 1 ./src/**/*.{hbs,json} ./build

echo "-> Build done"

This example is written in bash because it is a very short list of commands, each calling another smaller and much more focused script. You don’t have to use bash. Actually, if your script is more than a few lines long and requires some sort of logic, you might be better off using a more familiar programming language like JavaScript or Python or whatever you like.

Here’s the smallest possible example using Node.js as the runtime. This script can be invoked with ./scripts/node-env.

#!/usr/bin/env node
 * Print the current NODE_ENV

That’s it, that’s the idea. Just to recap:

  • Create a top-level scripts folder and add your scripts to it.
  • Scripts must not use a file extension. This will make it easier to change the language later.
  • Add the required shebang like to the file. 
  • Make sure the scripts are executable by running chmod +x scripts/<name of script>.
  • Invoke scripts by running ./scripts/name.
  • Only use NPM scripts to hook into the life cycle events like preinstall, postinstall, etc.
  • If you use one of the life cycle event hooks, just call one of your scripts.

Let me know what you think.

Photo: Unsplash

Categorized as Default

Paste JSON as Code

This is a quick post to recommend the Visual Studio Code extension Paste JSON as Code. The extension does exactly what it says on the name. You copy a piece of JSON, select paste as code, and the extension will generate and paste an interface for that JSON structure. It is super helpful when writing TypeScript.

Give it a try if you are using a statically typed language. It has support for a lot of them.

Categorized as Default

Your career needs a vision

Link: https://swizec.com/blog/your-career-needs-a-vision

It is well known the drunken sailor who staggers to the left or right with n independent random steps will, on the average, end up about √n, steps from the origin. But if there is a pretty girl in one direction, then his steps will tend to go in that direction and he will go a distance proportional to n. In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance √n, . In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not and therefore can only react to the current events as they happen. 

The Art of Science and Engineering, Richard Hamming

Categorized as Default

Bad names make you open the box

From the blog post “Bad names make you open the box” by Adam Zerner.

Imagine that you open the fridge. You see something labeled “brownie”. You eat it.

Then you hop in the car and start heading over to your friends house. But right as you merge on to the highway, you start feeling funny.

Turns out that the “brownie” label was a little misleading. It wasn’t a regular brownie. It was a pot brownie. There was something dangerous inside the brownie, but the label didn’t reflect that.

This is similar to poorly named functions with dangerous side effects. In both cases, if the thing in question can have dangerous side effects, you really want to make sure that it is reflected in the label. You can’t trust that people will read beyond the label. And even if you could, you wouldn’t want people to have to do that. You’d rather them be able to get the information they need from the label.

Adam Zerner
Categorized as Default

Eventually, You Will Need To Upgrade the Crypto

Every hashing and encryption algorithm will eventually become vulnerable and obsolete. Before that happens, you need to stay ahead and use something known to protect your sensitive data.

When the algorithm you are using becomes vulnerable, you must act quickly and replace it as soon as possible to minimize the exposure window. Replacing your current algorithm might be a daunting task if you did not prepare for change.

We must design systems for change.

Recently I had to upgrade an encryption library while maintaining the old library for backward compatibility. The new library gave us some flexibility; the old one did not have. However, the underlying algorithms stayed the same.

The idea was that whenever we needed to decrypt an old piece of data, we would re-encrypt using the new library. That way, we could gradually migrate to the new library without interruption. We needed a way to encode what library we were using with each piece of data to decide what library we needed to use to decrypt and if we needed to migrate that data to the new library format.

When presented with this challenge, I remembered how Django encoded passwords for database storage.


They encode the algorithm and other needed information to recreate the password hash into a string. Then, when a user enters a password, you can quickly parse this string and get all the required information to validate the hash and decide if the hash needs an upgrade.

It’s a straightforward idea that works great.

Quick aside to clarify: If you have a known vulnerable hash or encryption algorithm you will want to do something about it immediately and not wait for user action.

Specifically, in case of a vulnerable password hash the recommended technique is to rehash your current weak hashes with a secure hashing algorithm. Then, when the user logs in you can migrate their password hash to the new setup.

Finally, I wrote this post to remind you and me that we should add a version prefix whenever we need to store a password hash or encrypted data. That way, when we need to migrate, we are ready to do it.

Photo: Unsplash

Categorized as Default

Logging is important

Getting application logging right is crucial for every software project. When done correctly it makes the maintainers work a lot simpler. There’s no worst feeling than getting an error report and searching for logs to try to understand what happened and not finding a single entry. This is especially bad with difficult to reproduce production errors.

Like everything else, having good logs requires a little bit of thought and consistency. But is not that hard.

I’m not an expert on this subject. This post contains my notes on what has worked for me in the past in the context of backend web applications.

Use a real logger

Every language has a very simple way to send strings to stdout. That’s not a “real” logger.

In Node you have console.log() and in Python print(). These work great for simple scripts and print debugging. On a real production application, you want to use a “real” logger.

For Node, I prefer using pino. It’s fast, has great support, and is very easy to use.

What to log

  • Request/Response – You must log every request and response. For requests, I like to include all headers, except the Authorization header. For responses, I always include the HTTP status code and the response time. Both request and response include a unique request ID to correlate them.
  • App Errors – every catch in your code must include an error log describing what went wrong. Also whenever your app gets into a state that will result in an HTTP status code greater than 299 you should have some sort of application-level error log, describing why your request ended in an error.
  • Security events – Always log when a login fails, or when a user tries to access a page or an object is not supposed to. Always log password changes, email changes or any other security credentials change. Make sure you don’t log the actual password or sensitive information.
  • Business events – This is optional but it’s a good idea. If your app is a shopping cart, create a log entry every time you close a sale. The idea here is that you might want to create alerts based on these log events. If you usually see 100 sales per hour and suddenly see zero, it might be a good idea to take a look and understand what it’s going on.

A good log line

Having an unsearchable pile of logs is not useful. All my logs are in JSON format. This is very useful if you use a service that allows you to query JSON logs. I use AWS CloudWatch logs which is not perfect, but it does a decent job of helping me find what I’m looking for if I created the logs correctly.

Every log line must have the following.

  • Log level – Usually a number to indicate what kind of log it is. Info, error, and debug are the most common.
  • Message – This is the most important part of the log entry. Try to make this explicit and easy to understand for your future self or your fellow team members. Try to make messages unique so they are easy to search on your codebase. Having multiple error logs with the same message makes things harder than they have to be.
  • Group – All of my logs have a group field that use to indicate what part of the application is producing the log entry. Most of the time this group is used to indicate which specific file is creating the log entry. This facilitates searching.
  • Context – Application error logs usually require some context that will help you determine what is the problem without logging all of your application state. It is very important to include the minimum amount of data possible on every log entry but enough to determine what is going on. For example if your input validation failed it might be a good idea to log what exactly went wrong. For example if you only allow names of a certain length, it might be a good idea to log the length of the submitted name that produced the error.
  • Timestamp – Almost every logger adds a timestamp. Just make sure that the format used works for you and make sure you include the time zone or that your are very certain of what zone it’s being used.

Security concerns

If your system deals with PII or CHD, make sure you are not logging sensitive information. Pino and every decent logger have a feature to redact logs.

Also, make sure that you periodically check your logs for sensitive data. This is something not obvious and I’ve seen it go wrong multiple times.

Don’t log sensitive or secret information.

Final thoughts

I know logs are boring but they are an essential part of every app. A lot of thought is put into many aspects of software development but I rarely find people writing or discussing this subject. You should put some thought on your logging practices. You future self will thank you.

In summary:

  • Having decent logs will greatly improve the maintenance of your software.
  • Error logs are the most important kind of log.
  • Always include context on error logs.
  • Log every request and response and include status codes.
  • Make sure you don’t log sensitive information.
Categorized as Default