Introducing Laravel SEO Scanner

Introduction

As a developer using the Laravel framework for your web applications, it is important to optimize your site for search engines. The laravel-seo-scanner package can help with this by providing a quick and easy way to scan your Laravel application for common SEO issues. The package can check for missing or incorrectly used meta tags, broken links, response statuses, and performance-related factors such as file sizes.

At Vormkracht10, we have developed our own custom content management system (CMS). In short, customers can create content on the website using "content" models. However, due to the high level of customization allowed, users may sometimes create content that does not adhere to SEO standards. To address this, we wanted to provide both developers and customers with greater insight into the SEO scores of the content on their website. In addition to the default Google Lighthouse checks, we also consider other factors such as checking the robots.txt configuration and ensuring there are no dead links.

The package

The laravel-seo-scanner package enables us to proactively inform users about any "incorrect" content on their website, as well as offering opportunities for improvement. The package is designed to be fully configurable and offers the option to save scans in the database for future reference. It can also be used directly in the command line interface (CLI) if desired.

An example of scanning a single page on this website:

As you can see I still have some work to do haha. ;)

In our CMS, we have Content models that represent individual pages. We wanted to be able to save and display the SEO score for each page within the CMS. This functionality is only applied when a model is specified in the configuration file. For example, when you have a blog website - where each article represents a blog model from the database - you can simple add your blog model into the config. The results after each scan will then be saved in a separate database table that will be related to the configured model.

Different type of checks

The laravel-seo-scanner package includes a variety of checks that are divided into four categories: configuration, content, meta, and performance. These categories are intended to make it easier to identify and address specific types of SEO issues. Some checks, such as those related to content, may be more relevant to users who can manage their own content, while more technical checks like those related to performance may be more relevant to developers. You can choose which checks to enable or disable in the configuration file, and a full list of checks is available in the GitHub repository.

Configuration checks

Configuration checks are designed to ensure that your application is properly configured for search engines. These checks may include things like scanning the robots.txt file to ensure that indexing is allowed on the page being checked. These checks are typically relevant to developers, as they may require technical expertise to address.

Content checks

Content checks are focused on ensuring that the content of your pages is optimized for search engines. These checks may include things like checking for the correct use of alt tags, checking the length of your content, checking for broken links, and checking for the use of secure links (HTTPS). The results of these checks may be relevant to both developers and users, as users may be able to make some content-related improvements themselves.

Meta checks

Meta checks are focused on the meta tags used on your pages, and may include checks for the correct use of meta tags, the presence of an Open Graph image, and the use of words like "home" or "homepage" in the title. Some of these checks may be more relevant to developers, while others, such as the page title and Open Graph image, may be more relevant to users.

Performance

Performance checks are designed to ensure that your application is performing optimally for search engines. These checks may include things like measuring the Time To First Byte (TTFB), checking for appropriate response statuses, checking file sizes, and checking for the use of compression. The results of these checks are typically most relevant to developers, as they may require technical expertise to address. However, ensuring that your application is performing well can also have a positive impact on the user experience and may be of interest to users as well.

Adding your own checks

It's also possible to add custom checks to the laravel-seo-scanner package. We welcome pull requests with new check ideas, but if you prefer to keep them private that's also fine. To add your own checks to the package, follow these steps:

  1. Create a new class in your application that implements the Vormkracht10\Seo\Interfaces\Check interface. This will require you to define a check method that takes a Response object and a Crawler object as arguments, and returns a boolean indicating whether the check was successful or not.

  2. Add the Vormkracht10\Seo\Traits\PerformCheck trait to your class. This trait includes several properties and methods that are used by the laravel-seo-scanner package to manage and report on the results of the check.

  3. Define the properties of your class. These may include the name and priority of the check, the estimated time it takes to fix any issues found by the check, the weight of the check when calculating the overall score, and whether the check should continue after a failure. You may also want to define properties to store the actual and expected values of the check, and a reason for any failures.

  4. Implement the check method of your class. This should use the Crawler object to scan the HTML of the page being checked, and use any other validation techniques you see fit to determine whether the check is successful or not. You can use the $failureReason property to store a message explaining any failures.

<?php

namespace App\Support\Seo\Checks;

use Illuminate\Http\Client\Response;
use Symfony\Component\DomCrawler\Crawler;
use Vormkracht10\Seo\Interfaces\Check;
use Vormkracht10\Seo\Traits\PerformCheck;

class CanonicalCheck implements Check
{
    use PerformCheck;

    /**
     * The name of the check.
     */
    public string $title = "The page has a canonical tag";

    /**
     * The priority of the check (in terms of SEO).
     */
    public string $priority = 'low';

    /**
     * The time it takes to fix the issue.
     */
    public int $timeToFix = 1;

    /**
     * The weight of the check. This will be used to calculate the score.
     */
    public int $scoreWeight = 2;

    /**
     * If this check should continue after a failure. You don't
     * want to continue after a failure if the page is not
     * accessible, for example.
     */
    public bool $continueAfterFailure = true;

    public string|null $failureReason;

    /* If you want to check the actual value later on make sure
     * to set the actualValue property. This will be used
     * when saving the results.
     */
    public mixed $actualValue = null;

    /* If you want to check the expected value later on make sure
     * to set the expectedValue property. This will be used
     * when saving the results.
     */
    public mixed $expectedValue = null;

    public function check(Response $response, Crawler $crawler): bool
    {
        // Feel free to use any validation you want here.
        if (! $this->validateContent($crawler)) {
            return false;
        }

        return true;
    }

    public function validateContent(Crawler $crawler): bool
    {
        // Get the canonical tag
        $node = $crawler->filterXPath('//link[@rel="canonical"]')->getNode(0);

        if (! $node) {
            // We set the failure reason here so this will be showed in the CLI and saved in the database.
            $this->failureReason = 'The canonical tag does not exist';
            return false;
        }

        // Get the href attribute
        $this->actualValue = $node->getAttribute('href');

        if (! $this->actualValue) {
            // The failure reason is different here because the canonical tag exists, but it does not have a href attribute.
            $this->failureReason = 'The canonical tag does not have a href attribute';

            return false;
        }

        // The canonical tag exists and has a href attribute, so the check is successful.
        return true;
    }
}

  1. If desired, you can also create additional methods within your class to perform more specific checks or validation. For example, in the example code provided, the CanonicalCheck class includes a validateContent method to validate the presence and format of the canonical tag in the HTML.

  2. To use your custom check, you will need to add the base path of your check classes to the check_paths array in the configuration file. This will allow the laravel-seo-scanner package to discover and run your custom checks as