Some notes on Linting and Testing with node.js as well as some general thoughts on BDD & TDD frameworks.

[Most of these tools may be used when developing with plain javascript as well as with javascript frameworks such as React or Vue.]

All of the main browsers have extensions that can verify javascript. When testing client-side javascript components, these are probably the first place to start.

The notes will cover using the package manager npm but all of this can probably be done just as easily with yarn.

• Type Checking
• Linting
  • Disabling eslint rules
• Testing
  • Testing Frameworks
  • Assertion Libraries
  • Dependency Manipulation Libraries
  • Functional Testing Tools
    • CircleCI
    • GitHub Actions
    • GitLab
    • Travis CI
  • HTML Parsing
• Code Coverage Libraries
• Secrets Checking
• Dependency Checking
  • npm audit
  • OWASP
• Vulnerability Scanning
  • GitHub
  • Retire.js
  • Snyk.io
• Continuous Integration
• To Do

Type checking adds an extra level of validation when developing in javascript.

[This is the type of validation normally provided by a compiler.]

• Flow

To install:

npm install flow-bin -D

Update package.json as follows:

  "scripts": {
    "flow": "flow"
  },

To initialize:

npm run -s flow init

[This will create a .flowconfig file, which should be committed.]

To run:

npm run -s flow

Note that individual files must be annotated as follows in order to be scanned by flow:

// @flow

Status:

npm run -s flow status

Details:

http://flow.org/en/docs/

• JSLint
• JSHint
• ESLint

To install:

npm install eslint -D

Update package.json as follows:

  "scripts": {
    "eslint": "eslint"
  },

To set eslint options (if not set globally):

npm run -s eslint -- --init

To run:

npm run -s eslint .

To run and fix errors:

npm run -s eslint -- --fix .

Details:

http://eslint.org/docs/user-guide/command-line-interface

It is straightforward to disable eslint rules for specific lines of code:

// eslint-disable-next-line no-unused-vars
const routes = require("./routes.js")(app);

Reference:

http://eslint.org/docs/user-guide/configuring#disabling-rules-with-inline-comments

xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros, page 527

• AVA
• Mocha
• Jasmine
• Jest
• Tape

To install:

npm install mocha -D

Update package.json as follows:

  "scripts": {
    "test": "mocha"
  },

To run:

npm test

Depending upon your preferences for TDD versus BDD, any of the following libraries may be a good choice.

[Note that Ava, Jasmine and Jest include their own assertion libraries.]

TDD is an older and more established software practice, also hacker-friendly.

BDD is a related but more formal practice, which aims to align the tests with the functional specifications (which means that there need to be written functional specifications in place, perhaps more of a waterfall approach than an agile approach).

[My take on this is that BDD and TDD are really complementary disciplines. Both take best practices from Design by Contract and Use Cases (pre-conditions and post-conditions) and focus on putting Descartes before the horse (by writing the acceptance tests before the code - and only writing code once tests fail). This is in contrast to what I call 'Happy Path coding' which does not take exceptions or errors into consideration. My easy mnemonic for remembering which is which is 'T' for technical (such as testing for exceptions, errors or edge cases) testing and 'B' for business (business-related user acceptance criteria) testing.]

There are a number of BDD frameworks for languages such as Ruby (Cucumber, minitest) and Java (JGiven) but the focus here is on frameworks for javascript.

UPDATE: It seems that Cucumber has been ported to Javascript - Cucumber.js. There is at least one use of it in the wild - Botium BDD Samples (Botium is a testing framework for chatbots - it describes itself as "The Selenium for Chatbots").

• Assert
• code is a BDD assertion library
• Chai is a BDD / TDD assertion library
• Should.js

[Assert is node's native assertion library and does not need to be installed.]

To install:

npm install chai -D

• Mockery
• Proxyquire
• Rewire

To install:

npm install proxyquire -D

For more details:

http://github.com/thlorenz/proxyquire#usage

It is probably a good idea to also install Sinon:

• Sinon

Sinon describes itself as "Standalone test spies, stubs and mocks for JavaScript".

It works well with Proxyquire:

http://github.com/thlorenz/proxyquire/tree/master/examples/sinon

To install:

npm install sinon -D

• PhantomJS
• CasperJS
• Selenium WebDriver
• NightwatchJS
• WebdriverIO
• SuperAgent
• Chai HTTP
• Cypress

We will install Chai HTTP but Cypress is also worth considering (especially for Front-end Developers, as it can make them more Agile).

Cypress is a relative newcomer but shows promise. It plays well with Mocha and Chai.

Cypress is JavaScript-based which makes navigating a DOM straightforward. This may well obviate HTML Parsing, as described below. So, no Cheerio needed.

For more on Cypress, check out my Evergreen repo.

To install Chai HTTP:

npm install chai-http -D

For functional or performance testing, it is useful to be able to parse HTML responses (rather than grep for certain hard-coded text strings).

Either way, determining that the HTML response is correct makes for a very brittle and error-prone test.

For parsing and validating HTML, Cheerio should be useful - bearing in mind that it was designed for server-side HTML Parsing.

• Cheerio

To install:

npm install cheerio -D

Opinions differ on whether or not 100% coverage is a realistic or achievable goal, however code coverage is a very useful statistic for measuring software quality and how it changes over time.

[My opinion is that anything less than 70% code coverage is unacceptable. I have talked with many colleagues, and standards may change depending on the specific application or industry, with some sites requiring 80% coverage -- or even 100% coverage -- so I am NOT saying that 70% coverage is ideal, merely that 70% coverage is the bare minimum to start with, even if it results in some time lost in writing tests and/or refactoring. If your project is a few thousand lines of code or less, 100% coverage is probably the correct goal to shoot for.]

It's important to realize that 100% code coverage does NOT mean bug-free code!

A downside of testing and code coverage is that it often introduces brittle and difficult-to-maintain tests, all of which increase the number of lines of code. This should never be used as an excuse not to write tests, but it is something to bear in mind. For instance, it is a good idea to wait until any APIs have been finalized before spending too much time writing tests - as any API changes will lead to multiple code changes.

A balance must be maintained between writing tests and the burden of actually maintaining the test code, all while bearing in mind that tests - while valuable - do not deliver added functionality to the end user.

• Blanket (no longer maintained)
• JSCoverage (no longer maintained)
• Istanbul

To install:

npm install nyc -D

Update package.json as follows:

  "scripts": {
    "coverage": "nyc --reporter=text --reporter=html mocha"
  },

To run:

npm run -s coverage

It's quite easy for beginners or those who use IDEs or automated tools to check in secrets or certificates. Self-signed localhost certs are not really a concern, but certificates in general are something to keep an eye on. Secrets files, if they absolutely must be checked in, should be encrypted.

Submitting pull requests for code review can help, also there are repo-scanning options available - such as those from GitGuardian. But a carefully constructed .gitignore file is probably the first line of defence.

For private repos this is less of an issue, but for public repos it is something to be guarded against. It can be a real challenge to make secrets or certificates available to the CI/CD pipeline providers while keeping them secret from the public. Luckily most CI/CD providers have tooling available for this purpose, but it does tend to complicate the CI/CD pipeline.

Any build effort should include a tool to scan dependencies for known vulnerabilities.

Probably the first place to start is with npm (the node package manager) itself.

For an introduction to npm audit:

http://docs.npmjs.com/getting-started/running-a-security-audit

npm audit requires packages to have package.json and package-lock.json files.

To see what vulnerabilities it can detect:

$ npm audit

To see what fixes can be automatically made:

$ npm audit fix --dry-run

To apply the fixes:

$ npm audit fix

For a more comprehensive list of npm audit options:

http://docs.npmjs.com/cli/audit

[It is not clear whether or not this information relies on OWASP, probably best to use both.]

The Open Web Application Security Project (OWASP) sponsors a number of security projects, among them a dependency checker:

• OWASP Dependency-Check

[Platforms other than Java and .NET are supported via the Command Line tool.]

This tool may require that a Java Runtime Environment (JRE) be installed.

For more details:

http://jeremylong.github.io/DependencyCheck/general/thereport

OWASP also sponsors (along with others) the OWASP .NET Project and the OWASP Node js Goat Project.

While there are no doubt many other places to search for security vulnerabilities, OWASP is probably as good a place as any to start.

Any build effort should include tools to scan for vulnerabilities.

• Burp
• GitHub
• Retire.js
• Snyk.io

GitHub will scan your repos for vulnerabilities. They usually seem to be very much up-to-date. For most use cases, this is probably sufficient.

Retire.js publishes a list of the vulnerabilities that it can scan for, which is definitely worth a look.

Retire.js can be integrated as a Burp plugin.

Snyk offers reports, notifications and an attractive dashboard. It also offers a number of helpful integrations, such as GitHub and Travis/Jenkins. While Snyk will scan code whenever code is checked-in, it will also scan code on a regular basis as well. For a higher frequency of vulnerability scanning, a commercial license is recommended.

Snyk publishes a list of the vulnerabilities that it can scan for:

http://snyk.io/vuln

Snyk uses dependency manifests in order to determine the dependencies to scan:

http://support.snyk.io/getting-started/languages-support

This may be an issue to be aware of. For instance, note that for Golang repo testing, Snyk only supports a few of the possible manifest options offered by Golang's myriad dependency managers. And for Python, there will need to be a requirements.txt file. While very useful, it's worth bearing in mind that Snyk only scans projects under the control of a package manager - and uses the package manifest to do this. As the manifest may not actually reflect the installed packages, this needs to be taken into account (this should not be a concern in a CI or build pipeline).

For example, if the requirements.txt file contains Flask>=0.12.2 then Snyk will scan with the latest Flask (flask@1.0.2 at the time of writing). This may not reflect the locally installed version of the particular dependency (Flask here).

Snyk scans recursively, which is a very nice feature indeed. This means that that each package manifest will be scanned (and generate a report).

Snyk also offers a CLI option, but I believe you will need a Snyk account to use it.

For more details:

http://snyk.io/docs

This is really a best practice and should include linting and code coverage tests, as discussed above.

• BuildBot
• Jenkins
• Strider CD

For many uses, CIaaS (Continuous Integration as a Service) may be an attractive option. GitHub integration is usually relatively easy and painless, and often has a free tier.

• CircleCI
• GitHub Actions
• GitLab
• Travis CI

My experience has been mainly with GitHub as a source control provider, although I have also used GitLab professionally. If you are on GitLab, it's a good idea to stick with their CI tools as they are pretty good. For the other options discussed below, I will be considering them in terms of how well they integrate with GitHub.

CircleCI is relatively easy to use, and integrates well with GitHub.

For an example, check out my Circling repo.

Not surprisingly, GitHub Actions integrate well with GitHub. They are not yet the most easy to use (being still pretty much the new kid on the block) but will probably become easier to use eventually.

For an example with react.js, check out my ReactAWS repo.

For an example with vue.js, check out my VueRender repo.

GitLab features their own CI/CD pipelines and tools, which offer a pretty full slate of services.

If you are using GitLab for source control, also using GitLab for CIaaS seems like the way to go.

For more on GitLab, check out my GitLab repo.

Travis CI is easy to use and integrates well with GitHub such that a code commit can trigger an automated build as well as CI testing.

There are free as well as commercial options.

For an example, check out my RESTful Recipes repo.

• Investigate Cucumber [BDD framework for Ruby] and Gherkin (English-like DSL for expressing acceptance criteria)
• Investigate Cucumber.js [BDD framework for Javascript]
• Add some notes on Cypress
• Move Cypress notes to my Evergreen repo
• Add a note on when Snyk.io conducts vulnerability scans (on code check-in, as well as scheduled scans)
• Add a note on disabling eslint rules
• Add a note about GitHub Actions
• Add a note about GitHub and their vulnerability scans
• Add some notes about Secrets Checking
• Add more examples for CIaaS
• Investigate Serenity [BDD framework for validating use cases]
• Investigate SpecFlow [BDD framework that describes itself as "Cucumber for .NET"]