Automated frontend perfomance test with lighthouse for JHipster -

The JHipster team always tries to generate code that performs well. For the backend code you can decide to use Gatling to generate perfomance tests. Most of the times Matt Raible is doing a lighthouse test for each release manually, but there is no automated regression testing for a generated application.

When choosing Cypress as end-2-end testing framework one could use cypress-audit to automate the manual lighthouse test. This will be beneficial to the JHipster development to identify regressions and for JHipster users as they can test the frontend perfomance more easily. Furthermore cypress-audit supports accessibility testing via Pa11y.

This post describes how to integrate cypress-audit into a JHipster application to execute lighthouse and pa11y testing via GitHub actions.

Install cypress-audit

For the general installation the official documentation is very good. For more details you should read this blog post.

npm install --D cypress-audit lighthouse

Setup cypress-audit

In src/test/javascript/cypress/plugins/index.ts you find the plugin definitions and need to setup cypress-audit.

index.ts

const { lighthouse, pa11y, prepareAudit } = require('cypress-audit'); 1
const fs = require('fs'); 2
const ReportGenerator = require('lighthouse/lighthouse-core/report/report-generator'); 3

module.exports = (on, config) => {
  on('before:browser:launch', (browser, launchOptions) => {

    prepareAudit(launchOptions); 4
    if (browser.name === 'chrome' && browser.isHeadless) {
      launchOptions.args.push('--disable-gpu');
      return launchOptions;
    }
  });

  on('task', {
    lighthouse: lighthouse((lighthouseReport) => { 4
      fs.writeFileSync('build/cypress/lhreport.html', ReportGenerator.generateReport(lighthouseReport.lhr, 'html')); 5
    }),
    pa11y: pa11y(), 6
  });
};

Import cypress audit modules
Import node fs module to write report to dsik
Import lighthouse report generator to generate html report
Register custom lighthouse task
Use the report generator to create and save a html report to build output folder
Register custom pa11y task

In src/test/javascript/cypress/support/commands.ts you need to import the cypress audit commands.

commands.ts

import 'cypress-audit/commands';
export {};

Create audit specification

After the setup is complete you need to audit the JHipster main page. To do so create a new file in src/test/javascript/cypress/cypress-audit.spec.ts.

cypress-audit.spec.ts

describe('Audits', () => {
    beforeEach(() => { 1
      cy.visit('/');
    });

    it('pa11y audits', () => { 2
        cy.pa11y();
    });

    it('lighthouse audits', () => {
      const customThresholds = {  3
        performance: 70,
        accessibility: 90,
        seo: 90,
        'best-practices': 90,
        pwa: 0,
      };

      const desktopConfig = { 4
        extends: 'lighthouse:default',
        formFactor: 'desktop',
        screenEmulation: {disabled: true}
      };
      cy.lighthouse(customThresholds, desktopConfig); 5
    });
});

Open the main page before each audit
Execute accessibility audit via pa11y
Define custom thresholds for successful audit
Define lighthouse configuration to emulate desktops
Execute performance audit via lighthouse

Exexute the tests

Start the application in production mode to have the frontend minified and optimized. Afterwards run the end-2-end tests:

docker-compose -f src/main/docker/postgresql.yml up -d
./gradlew -Pprod
npm run e2e

This will execute the defined audits and fail if any of the defined thresholds is not met.

HTML Report

The well-known lighthouse html report is created under build/cypress/lhreport.html and can be used see a detailed performance analysis even when all thresholds are met.

Tip:

In comparison to the default lighthouse configuration when using e.g. chromium, the lighthouse cli emulates a 'mid-tier mobile phone' while chromium emulates a 'high-end desktop' by default. This is why the performance rating is not as good as you might expect.

For comparison the generated vue.js frontend scores a much better performance rating with the same configuration and slowdown.

To change the slowdown multiplier to your needs to can adapt the config like:

const desktopConfig = {
    extends: 'lighthouse:default',
    formFactor: 'desktop',
    throttling: {
        cpuSlowdownMultiplier: 1
    },
    screenEmulation: {disabled: true}
};

Lighthouse report for default angular application

Lighthouse report for default vue application

Execute on CI

JHipster helps you create a working GitHub actions configuration. Execute jhipster ci-cd and select GitHub Actions. This will create a GitHub Action which executes backend, frontend and end-2-end tests.

Conclusion

Cypress audit enabled regression testing and continous frontend performance testing for JHipster applications using the already created cypress test setup. If you want to try it yourself you find the source code on GitHub.

jhipster cypress lighthouse java angular vue