Performance Testing using Lighthouse CI Action Performance matters. Client-side, middleware, database, server-side whatever hop you are talking about, performance matters. Google Chrome’s Audit (Lighthouse in Chrome DevTools) helps you to improve the quality of the web page by validating critical factors like SEO, Accessibility, Performance, Progressive Web App (PWA), and Best Practices.
Running audits every time is cumbersome. You can automate running audits in CLI (command line interface) mode. There are recipes for Continuous Integration using gulp.
But now the CI process is hassle-free using Lighthouse CI. It is an open source project available at GitHub where you can run the audits continuously running, asserting, saving, and viewing Lighthouse audit regression results.
In this blog post, I am going to demonstrate how you can implement GitHub Actions to run the Lighthouse audits for the demo page using Lighthouse CI Action.
Table of Contents
What is GitHub Actions?
GitHub Actions is an API for cause and effect on GitHub: orchestrate any workflow, based on any event, while GitHub manages the execution, provides rich feedback, and secures every step along the way. With GitHub Actions, workflows and steps are just code in a repository, so you can create, share, reuse, and fork your software development practices.
Source https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/
Running Lighthouse CI Action is absolutely free and no infrastructure required. Thanks to Treo and Google.
Prerequisites
- GitHub Account.
- You can create it for free in here.
- GitHub Repository with demo web page.
- Clone my repository here.
Let us create a simple HTML page in GitHub. Alternatively, you can clone/fork my GitHub repository here.
Steps to create a new repository
- Login into your GitHub account.
- Create a new repository by visiting this link.
- Enter a valid repository name e.g.
Lighthouse-CI-Demo-Repo
- Enter optional Description
- Initialize your repository with README by checking
Initialize this repository with a README
- Hit
Create Repository
button
Steps to create a HTML page
- Click on
Create new file
- Name your file as
index.html
- Copy and paste the below code
<!DOCTYPE html>
<html>
<title>Lighthouse CI Demo</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="Expires" content="6000" />
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Raleway">
<style>
body,h1 {font-family: "Raleway", sans-serif}
body, html {height: 100%}
.bgimg {
background-image: url('BG-AfterCompression.jpg');
min-height: 100%;
background-position: right;
background-size: cover;
backgorund-color: black;
}
</style>
<body>
<div class="bgimg w3-display-container w3-animate-opacity w3-text-white">
<div class="w3-display-topleft w3-padding-large w3-xlarge">
Your Logo Here
</div>
<div class="w3-display-middle">
<h1 class="w3-jumbo w3-animate-top">COMING SOON</h1>
<hr class="w3-border-grey" style="margin:auto;width:40%">
</div>
<div class="w3-display-bottomleft w3-padding-large">
Lighthouse CI Demo - NaveenKumar Namachivayam - QAInsights.com </div>
</div>
</body>
</html>
- Click on
Commit new file
button - Download this background image and upload it to your repository.
Now go to Settings
page of your repository, scroll down to GitHub Pages
section, select the Source
as master branch
Click on Choose a theme
button, select a theme of your choice and hit Select a theme
button and then hit Commit changes
Go back to Settings
page and scroll down to GitHub Pages
section, you could see the site URL e.g. https://qainsights.github.io/Lighthouse-CI-Demo-Repo/.
Useful GitHub Repositories
* Lighthouse CI
* Lighthouse CI Action
* Lighthouse CI Action Demo
Performance Testing using Lighthouse CI Action
Launch the URL which will open the web page with the background image and text. Let’s create an Action for Lighthouse Audit.
Click on Actions
menu in your repository page as shown below and then click on Setup a workflow yourself
button.
Enter lighthouse ci action
in the Marketplace
search text box and hit enter.
Click on Lighthouse CI Action
, you could see the installation code snippet. Copy the snippet and click on Start Commit
button > Commit new file
button. This will add the main.yml
to your repository.
- name: Lighthouse CI Action
uses: treosh/lighthouse-ci-action@v1
main.yml
file will have the GitHub actions to be performed based on the instructions provided.
Go to main.yml
and paste the below snippet and commit it. You need to replace the URL with your GitHub page link.
name: Lighthouse
on: push # For every push, this action will be triggered
jobs:
lighthouse:
runs-on: ubuntu-latest #OS
steps:
- uses: actions/checkout@v1
- name: Audit URLs using Lighthouse
uses: treosh/lighthouse-ci-action@v2
with:
urls:
https://qainsights.github.io/Lighthouse-CI-Demo-Repo/
- name: Save results
uses: actions/upload-artifact@v1
with:
name: lighthouse-results
path: '.lighthouseci' # This will save the Lighthouse results as .json files
Now the GitHub actions set up is complete. It is time to run the Lighthouse audits for the GitHub page.
Open index.html
and make some changes e.g. add <h1
class="w3-jumbo w3-animate-top">Lighthouse Audit Run #1</h1>
and commit the changes.
After committing the changes, head to Actions
page, you can see the workflow being queued.
Click on the workflow you would like to view where you can see the logs. To download the results, click on Artifacts
and download the zip file.
How to view the Lighthouse Audit results?
Extract the downloaded results where you can see the JSON file. Go to Lighthouse Report Viewer
and drag and drop the JSON file.
You can see the gist here.
You can check all the workflows here.
As you see the Accessibility score is 84. Let’s improve that score by seeing the recommendation from the Lighthouse Audit report. The recommendation is html tag does not have a [lang] attribute. Let’s add the attribute as ‘en’ and commit the changes.
<html> element does not have a [lang] attribute
Replace <html> tag with `<html lang=”en”> and commit the changes. After the commit, Lighthouse CI Action will be triggered and the results will be available in the workflows.
As you see the latest report, the Accessibility Score is 100. There is still a room for SEO and Performance improvement.
Pros
- GitHub Actions are free
- Lighthouse CI Action is free
- No infrastructure is required to run the tests
- You can store the results in LHCI server
- Complete control over Lighthouse and its CI
- Performance Budgets
- Assertions
For advanced usage, you can check the marketplace.
Conclusion
Measuring performance in all the layers is critical. Lighthouse Audits is one of the tool which will help you to improve the overall quality of the page. Lighthouse CI and its Action is very useful for the regression performance tests. With the help of Treo, GitHub Actions and Google, you could run the performance regression tests without any financial investments.
Thanks for the wonderful Lighthouse Action CI process. I followed all the steps, however the job is successfull but i do not see Download Artifacts link
Please share the error screenshot.
Hi Naveen,
Thanks for the blog. Currently i am using Web Page Test with some scripting inside it to invoke some user actions/Navigations, could you please share your thought on implementing these using lighthouse?
Basically many of the times widgets will be loaded on call, so would like to get individual widget responses too. Using web page test i can achieve it by mimicking the use actions using javascript.
Appreciate your help!
Thanks,
Naveen
Hi, Could you please provide what metrics you are capturing using Web Page test? Thanks!
Let’s go with your example, making change to an html will trigger the test and it will reload the page, during on-load lighthouse captures metrics and display on dashboard.
In my case, i am on an enterprise application and when i click on a link only portion will get loaded with the content and remaining portions will stay calm on the UI. So my goal here is to capture the response time to load that particular portion on UI. For this i’ve used some basic commands like execAndWait in web page test to achieve these. Would it be possible to mimic the same using lighthouse?
Thank you for the share. I followed steps mentioned above and it works.
Thanks for trying :)
Hi Naveen,
In one of my application it need authentication (username/password).
Without it, we cannot proceed .
How can we insert authentication (username/password) in above mentioned steps in blog .
Could you please help!!
Hi Ankit, Lighthouse evaluates page performance based on variety of factors. It is not an automation tool. If you want to evaluate performance, you can disable the authentication and run the evaluation. Or you can run the evaluation manually by invoking the Dev tools in Chrome browser.
Hi Naveen,
I am trying to use private lhci server which is hosted on my local using docker desktop for windows.
I have completed the setup using lhci wizard and providing the server and token details in main.yml file. But during upload , job is giving below error.
Could you please help on this? Is it because lhci server is hosted on my localhost
Error:
FetchError: request to http://localhost:9001/v1/projects/lookup failed, reason: connect ECONNREFUSED 127.0.0.1:9001
at ClientRequest. (/home/runner/work/_actions/treosh/lighthouse-ci-action/v2/node_modules/node-fetch/index.js:133:11)
at ClientRequest.emit (events.js:210:5)
at Socket.socketErrorListener (_http_client.js:406:9)
at Socket.emit (events.js:210:5)
at emitErrorNT (internal/streams/destroy.js:92:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
Error: LHCI ‘upload’ failed to upload to LHCI server.
at processTicksAndRejections (internal/process/task_queues.js:80:21)