How to monitor the proper functioning of a website

In times where it is possible to instantly access information, the website is the main point of communication with the market for every company. The website is often the place where the crucial first impression is built. It generates leads, sells, provides customer service, gets in touch with the media, recruits etc.

It is the company’s website that the traffic from advertising campaigns gets directed to—and not only from the online ones. After watching an advertisement on television or seeing one in the press, the recipients naturally type in the website’s address—or even the name of the company itself—on their smartphone or computer.

Faulty operation of the company’s website generates more losses than a broken telephone or the need to temporarily close the company’s store. The users that we tried so hard to attract to the site will not be converted (into leads, orders, newsletter subscriptions, etc.) but they will also get a negative impression and will most likely immediately switch to the website of the competition.

You can pause your online campaign to minimize losses—although you won’t be able to stop organic traffic from the search engine. But printed press advertisements cannot be undone, just as outdoor posters can’t be removed immediately. The only solution is to immediately remove the problem—and this is dependent on the immediate detection of it.

Website monitoring

Services for monitoring the availability and proper functioning of websites and web applications have existed for almost as long as the websites themselves. And they develop together with the technologies used on the websites. Their main task is to detect problems and inform the website owners. So that they find out about every failure and malfunction first—before any of the users notice.

In the history of the monitoring test, in addition to the basic information about each failure (detection time, duration, type), you can find a lot of additional information to help you analyze the incident and search for its causes. These include HTTP headers, HTML source, screenshots, HAR files etc.

An additional application of monitoring is to verify availability and performance on a monthly basis—for example, to compare with the levels guaranteed in SLAs.

Website unavailability

The total unavailability of the website most often results from infrastructural problems. It may be a hardware (e.g. disk), software (e.g. system configuration error), network (no access to the Internet) fault. These may include force majeure, negligence and deliberate harmful actions (e.g. a DDoS attack).

In most cases, the hosting service providers are immediately aware of failures because they use their own monitoring. But it is still worth submitting a ticket—in case the failure was extensive and the provider carried out repairs on the first come, first served basis. Or if the provider’s monitoring didn’t work.

This type of failure cannot be avoided—they happen even to the largest. But you can minimize the risk by using cloud solutions, running additional servers or even secondary data centers.

Unavailability of the website may also be the result of exceeding the server’s parameters. The hosting service must be adapted to the popularity of the website. The higher the traffic, the more resources are consumed. Of course, it is very important to optimize the website’s engine so that it doesn’t consume unnecessary memory or CPU time.

However, it sometimes happens that the brand gains sudden, unexpected popularity. For example, after a successful PR action which resulted in many publications in the media. If the infrastructure is not prepared for a jump in traffic of several thousand per cent, the page won’t load for a large number of users.

How to detect website’s unavailability

Total unavailability of the site is the easiest to detect—even the simplest monitoring services can handle it. They operate by establishing a connection to the website and checking the server response code. If the answer is “200 OK”, then everything is fine. A different response (e.g. “500 Internal Server Error”, “503 Service Unavailable”), or no response within e.g. 10 seconds means a malfunction. There are also redirection codes (“301 Moved Permanently”, “302 Found”), which don’t mean failure—unless there are too many of them and redirections get looped.

Wrong content of the website

A database failure, programmer error or hacker attack can cause the website to not display what it should. For example, only half of the page, blank templates without content, or a third-party page inserted by a hacker. This is a worse situation than the usual unavailability of the site, because users may think that the wrong content they see is displayed as intended by the website owner.

How to detect wrong website content

The basic monitoring of accessibility won’t notice this type of failure. It will register the correct server response (“200 OK”) and decide that the website is working properly. Thus, you must use the tests that verify the content of the page—checking for the presence (or absence) of a defined phrase or code snippet. You can configure such a test to search the website, for example, for a fragment of text displayed in the footer coming from the database (i.e. not hardcoded into the template). The lack of such text on the page will mean that the page has not been fully loaded or the database server has failed.

Website slowdown

Users—especially mobile ones—are less and less patient. It’s enough that the site doesn’t load within seconds for them to give up and—most often—return to the search results and switch to the website of the competition. Even a website well-optimized for fast loading can be very easily “damaged”—for example by publishing too large, uncompressed graphics. Of course, problems with the hosting server can also increase the page loading speed.

In the long run, a slow website may also have a negative impact on the position in the search results. For a long time, Google has been treating page speed as one of the ranking factors.

How to detect website slowdowns

Monitoring page loading times is a feature that is not offered by all monitoring service providers. Mainly because it is complicated and resource-intensive. Such monitoring requires the page to be fully loaded every time—including all embedded elements. This means all CSS styles, fonts, graphics and JavaScript.

Configuring such a test consists of setting the limit of acceptable page loading time. If it is exceeded, failure will be registered and the website owner will receive a notification. Of course, assuming that the problem gets confirmed from several locations.

Key feature malfunction

A situation that happens quite often: a website is available, it loads in full and quite quickly, but one of the key features (e.g. adding a product to the shopping cart) doesn’t work. Without proper monitoring of such a defect, the owner of the website will probably find out only after watching the sudden drop in sales. It is less likely that one of the users will care so much about using a feature that they report a problem.

How to detect key feature malfunctions

The most advanced website monitoring function is the monitoring of processes by scenarios. Such a test consists of cyclically pretending to be a real user using a real browser and executing a sequence of actions according to a predefined scenario. Filling out forms, clicking buttons, waiting for an item to appear, etc.

If a problem occurs at any of the steps in the scenario—for example, lack of the expected element or incorrect response to a given activity—the monitoring will register and report failure, providing information at which step it occurred and taking a screenshot.

Expiration of the SSL certificate or domain

Nowadays, securing the website by encrypting all communication (https instead of http) is a must. But it involves the need to renew the SSL certificate of the server and remember its expiration date. Of course, the certificate provider informs you of the impending expiry date. But sometimes when buying a certificate, we use an e-mail address that we rarely check. Or such notifications go to the spam folder and they escape our attention.

Similarly, you can miss the deadline for renewing your domain. Regardless of whether the certificate or domain has expired, from the user’s point of view, the website will be unavailable.

How to monitor the expiration of the SSL certificate and domain

An alert when the certificate or domain expires is a signal that a critical situation has occurred. Of course, it is much better to prevent and avoid than to react. Therefore, the monitoring of domain and SSL certificates checks the expiration date and informs the owner in advance (several times) that the expiration date is approaching.

What else can and should you monitor

Monitoring services can also detect other problems, for example:

• unauthorized changes to important files created as a result of the actions of hackers
• the appearance of the website on blacklists (browsers block the opening of such pages, displaying a menacing warning in red)
• unintentional blocking of access for search engine robots, which may result in quite a fast disappearance of the website from Google
• incorrect operation of servers other than WWW—database, mail, files, VoIP communication, etc.

How to use monitoring services

Configuration

The configuration of the basic test that monitors the availability of the website is very simple. Just enter the URL of the website and it’s done. For the content monitoring test, you must additionally enter the text (or code), the presence of which is to be checked on the website. Configuring the loading speed monitoring basically involves setting an acceptable loading time limit. The tests that check for the expiration of domains and SSL certificates more or less configure themselves—here you also simply need to enter the URL.

It is more difficult to monitor forms and processes. Here, you have to define all steps of a scenario, appropriately indicating the elements on the page. But you can always use the help of the monitoring company’s customer service. Consultants do it every day and the configuration of such a scenario will only take a moment.

Notifications

The basic channel for receiving information about failures is e-mail. But not everybody has their mobile e-mail client set to check for new messages every minute and play an alarming sound when the monitoring e-mail is received.

That’s why you should enable SMS alerts. So as not to be woken up at two in the morning—when there is no need, because the administrator also receives the alerts—you can specify the hours at which the text messages may be sent.

In addition to an immediate alert, when failure is detected, you can set additional notifications—for example, after the first or every 5 minutes of failure, or after its fixed.

Reports

In the user panel, in the history of each test, you can find detailed information about the selected failure, as well as get a broader view of the availability and performance of your website—e.g. on a monthly or yearly scale. Periodic reports—weekly or monthly—can also be generated automatically and sent via e-mail.

If multiple websites are monitored on your account, you can create sub-accounts—for co-workers or clients—by giving them access to selected tests.

Integrations

Thanks to various plugins, modules and extensions, the monitoring system can be used inside your CMS. This allows you to use it without logging into a separate application.

Moreover, the API can be used to integrate monitoring with your own application—which will display or analyze availability and performance data itself.

Summary

Monitoring of availability and proper functioning is indispensable for any commercial website. It doesn’t matter if it’s a company website, product page, landing page, online store or web application. In each of these cases, failure or malfunction translates into specific financial and image losses. Monitoring allows you to react quickly and minimize negative consequences.

It’s worth investing some time into configuring your monitoring tests. And the subscription fee for the use of monitoring services will certainly be lower than the costs of the missed failure.