Performance Testing using Lighthouse CI Action

Performance Testing using Lighthouse CI Action Performance matters. Client-side, middleware, database, server-side whatever hop you are talking about, performance matters. Google Chrome’s Audit (Lighthouse in Chrome DevTools) helps you to improve the quality of the web page by validating critical factors like SEO, Accessibility, Performance, Progressive Web App (PWA), and Best Practices.

Performance Testing using Lighthouse CI
Performance Testing using Lighthouse CI

Running audits every time is cumbersome. You can automate running audits in CLI (command line interface) mode. There are recipes for Continuous Integration using gulp.

But now the CI process is hassle-free using Lighthouse CI. It is an open source project available at GitHub where you can run the audits continuously running, asserting, saving, and viewing Lighthouse audit regression results.

In this blog post, I am going to demonstrate how you can implement GitHub Actions to run the Lighthouse audits for the demo page using Lighthouse CI Action.

What is GitHub Actions?

GitHub Actions is an API for cause and effect on GitHub: orchestrate any  workflow, based on any event, while GitHub manages the execution,  provides rich feedback, and secures every step along the way. With  GitHub Actions, workflows and steps are just code in a repository, so  you can create, share, reuse, and fork your software development  practices. 

Source https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/

Running Lighthouse CI Action is absolutely free and no infrastructure required. Thanks to Treo and Google.

Prerequisites

  1. GitHub Account.
    • You can create it for free in here.
  2. GitHub Repository with demo web page.
    • Clone my repository here.

Let us create a simple HTML page in GitHub. Alternatively, you can clone/fork my GitHub repository here.

Steps to create a new repository

  • Login into your GitHub account.
  • Create a new repository by visiting this link.
  • Enter a valid repository name e.g. Lighthouse-CI-Demo-Repo
  • Enter optional Description
  • Initialize your repository with README by checking Initialize this repository with a README
  • Hit Create Repository button

Steps to create a HTML page

  • Click on Create new file
  • Name your file as index.html
  • Copy and paste the below code
<!DOCTYPE html>
<html>
<title>Lighthouse CI Demo</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="Expires" content="6000" />
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Raleway">
<style>
body,h1 {font-family: "Raleway", sans-serif}
body, html {height: 100%}
  .bgimg {
  background-image: url('BG-AfterCompression.jpg');
  min-height: 100%;
  background-position: right;
  background-size: cover;
  backgorund-color: black;
}
</style>
<body>

<div class="bgimg w3-display-container w3-animate-opacity w3-text-white">
  <div class="w3-display-topleft w3-padding-large w3-xlarge">
    Your Logo Here
  </div>
  <div class="w3-display-middle">
    <h1 class="w3-jumbo w3-animate-top">COMING SOON</h1>    
    <hr class="w3-border-grey" style="margin:auto;width:40%">
  </div>
  <div class="w3-display-bottomleft w3-padding-large">
Lighthouse CI Demo - NaveenKumar Namachivayam - QAInsights.com  </div>
</div>

</body>
</html>

Now go to Settings page of your repository, scroll down to GitHub Pages section, select the Source as master branch

Click on Choose a theme button, select a theme of your choice and hit Select a theme button and then hit Commit changes

Go back to Settings page and scroll down to GitHub Pages section, you could see the site URL e.g. https://qainsights.github.io/Lighthouse-CI-Demo-Repo/.

Useful GitHub Repositories 

* Lighthouse CI

* Lighthouse CI Action

* Lighthouse CI Action Demo

Performance Testing using Lighthouse CI Action

Launch the URL which will open the web page with the background image and text. Let’s create an Action for Lighthouse Audit.

Click on Actions menu in your repository page as shown below and then click on Setup a workflow yourself button.

Setup Workflow
Setup Workflow

Enter lighthouse ci action in the Marketplace search text box and hit enter.

Lighthouse CI Action
Lighthouse CI Action

Click on Lighthouse CI Action, you could see the installation code snippet. Copy the snippet and click on Start Commit button > Commit new file button. This will add the main.yml to your repository.

- name: Lighthouse CI Action
  uses: treosh/lighthouse-ci-action@v1

main.yml file will have the GitHub actions to be performed based on the instructions provided.

Go to main.yml and paste the below snippet and commit it. You need to replace the URL with your GitHub page link.

name: Lighthouse
on: push # For every push, this action will be triggered
jobs:
  lighthouse:
    runs-on: ubuntu-latest #OS
    steps:
      - uses: actions/checkout@v1
      - name: Audit URLs using Lighthouse
        uses: treosh/lighthouse-ci-action@v2
        with:
          urls:
            https://qainsights.github.io/Lighthouse-CI-Demo-Repo/
      - name: Save results
        uses: actions/upload-artifact@v1
        with:
          name: lighthouse-results
          path: '.lighthouseci' # This will save the Lighthouse results as .json files

Now the GitHub actions set up is complete. It is time to run the Lighthouse audits for the GitHub page.

Open index.html and make some changes e.g. add <h1 class="w3-jumbo w3-animate-top">Lighthouse Audit Run #1</h1> and commit the changes.

After committing the changes, head to Actions page, you can see the workflow being queued.

All Workflows
All Workflows

Click on the workflow you would like to view where you can see the logs. To download the results, click on Artifacts and download the zip file.

Download Artifacts
Download Artifacts

How to view the Lighthouse Audit results?

Extract the downloaded results where you can see the JSON file. Go to Lighthouse Report Viewer and drag and drop the JSON file.

You can see the gist here.

You can check all the workflows here.

Clone my repository

Lighthouse Audit Report
Lighthouse Audit Report

As you see the Accessibility score is 84. Let’s improve that score by seeing the recommendation from the Lighthouse Audit report. The recommendation is html tag does not have a [lang] attribute. Let’s add the attribute as ‘en’ and commit the changes.

<html> element does not have a [lang] attribute

Replace <html> tag with `<html lang=”en”> and commit the changes. After the commit, Lighthouse CI Action will be triggered and the results will be available in the workflows.

As you see the latest report, the Accessibility Score is 100. There is still a room for SEO and Performance improvement.

Performance Testing using Lighthouse CI Action
Performance Testing using Lighthouse CI Action

Pros

  • GitHub Actions are free
  • Lighthouse CI Action is free
  • No infrastructure is required to run the tests
  • You can store the results in LHCI server
  • Complete control over Lighthouse and its CI
  • Performance Budgets
  • Assertions

For advanced usage, you can check the marketplace.

Conclusion

Measuring performance in all the layers is critical. Lighthouse Audits is one of the tool which will help you to improve the overall quality of the page. Lighthouse CI and its Action is very useful for the regression performance tests. With the help of Treo, GitHub Actions and Google, you could run the performance regression tests without any financial investments.

About the Author

10 thoughts on “Performance Testing using Lighthouse CI Action”

  1. Thanks for the wonderful Lighthouse Action CI process. I followed all the steps, however the job is successfull but i do not see Download Artifacts link

    Reply
  2. Hi Naveen,

    Thanks for the blog. Currently i am using Web Page Test with some scripting inside it to invoke some user actions/Navigations, could you please share your thought on implementing these using lighthouse?

    Basically many of the times widgets will be loaded on call, so would like to get individual widget responses too. Using web page test i can achieve it by mimicking the use actions using javascript.

    Appreciate your help!

    Thanks,
    Naveen

    Reply
      • Let’s go with your example, making change to an html will trigger the test and it will reload the page, during on-load lighthouse captures metrics and display on dashboard.

        In my case, i am on an enterprise application and when i click on a link only portion will get loaded with the content and remaining portions will stay calm on the UI. So my goal here is to capture the response time to load that particular portion on UI. For this i’ve used some basic commands like execAndWait in web page test to achieve these. Would it be possible to mimic the same using lighthouse?

        Reply
  3. Hi Naveen,

    In one of my application it need authentication (username/password).
    Without it, we cannot proceed .

    How can we insert authentication (username/password) in above mentioned steps in blog .
    Could you please help!!

    Reply
    • Hi Ankit, Lighthouse evaluates page performance based on variety of factors. It is not an automation tool. If you want to evaluate performance, you can disable the authentication and run the evaluation. Or you can run the evaluation manually by invoking the Dev tools in Chrome browser.

      Reply
  4. Hi Naveen,

    I am trying to use private lhci server which is hosted on my local using docker desktop for windows.
    I have completed the setup using lhci wizard and providing the server and token details in main.yml file. But during upload , job is giving below error.
    Could you please help on this? Is it because lhci server is hosted on my localhost

    Error:
    FetchError: request to http://localhost:9001/v1/projects/lookup failed, reason: connect ECONNREFUSED 127.0.0.1:9001
    at ClientRequest. (/home/runner/work/_actions/treosh/lighthouse-ci-action/v2/node_modules/node-fetch/index.js:133:11)
    at ClientRequest.emit (events.js:210:5)
    at Socket.socketErrorListener (_http_client.js:406:9)
    at Socket.emit (events.js:210:5)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    Error: LHCI ‘upload’ failed to upload to LHCI server.
    at processTicksAndRejections (internal/process/task_queues.js:80:21)

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Hamster - Launch JMeter Recent Test Plans SwiftlyDownload for free
+
Share via
Copy link