Do all your synthetic monitors include a content check? Why not? Content checks are free with all our monitor types, but for the most part, Uptrends users underutilize content checks. In this article, we talk about why content checks are important for your monitoring, and we touch on some tips to help you pick the content checks that work best for you.
What is a content check?
A content check is a simple check conducted by the monitor on the page or response contents verifying a value (a word, a number, a phrase) is or is not in the content. For example, if you know your page should include the word “Welcome” followed by a name, you can verify if the word is present on the page or within the response using a regular expression.
Why use a content check?
A successful response (200 ok) isn’t a guarantee that the user landed on the correct page or the page completely loaded. For example, let’s say you’ve set up a page redirect incorrectly or someone else mistakenly removed the redirect from a page, so instead of directing users to your newest product page, the user gets the old page or some other random page. If the monitor includes a content check that looks for something specific to the new products page, Uptrends generates an error when the monitor can’t find the content. With the alert caused by the failed content check, you would know about the problem. Without the content check, the monitor will get a successful result, and you wouldn’t know that your server is sending the wrong content.
The above is just one example of a situation where you can get a false positive. Let’s look at a few more.
Incomplete content
If your server or CDN (Content Delivery Network) doesn’t respond with the complete page content, you may still get a successful code, but the page isn’t usable. The page loads fine, but the user gets a partial page such as the header and footer with nothing between, perhaps they get a completely blank page or that one important page element the user needs doesn’t appear on the page. The result is a 200 ok, but what can you do with a blank page other than bounce to the competition? You would want to know about that, wouldn’t you?
Wrong content
Just like the incomplete content, your server may respond with the wrong content due to a redirect or a database glitch for example. Again, unless there is a content check, you wouldn’t know about it until your users start complaining.
DNS hijacking, spoofing, or poisoning
If a hacker manages to hijack your DNS records and reroute your URL to a different IP address, you wouldn’t know without a content check. Not catching a DNS hijacking right away can compromise your site and your users’ accounts. A simple content check can capture some of these hacks. You might want to take it a step further using a DNS or SSL monitor to protect yourself from these issues further.
Types of content checks
You have options when it comes to content checks. You can check for an exact match or use a regular expression to find content based on patterns matches.
String literals or an exact text match
You can check for any phrase, word, or sequence of characters expected on the page. For example, you may want to search for the phrase “open an account” on your page (see Figure 1), or by adding an exclamation point, you can check to make sure a phrase isn’t on your page like so, “!open an account” (see Figure 2). Note: the check is not case sensitive. Use a regular expression for a case sensitive check.
Figure 1: Check to make sure a word or phrase appears on the page.
Figure 2: Use “!” to make sure something doesn’t appear on the page.
Pattern matching a.k.a. regular expressions
A regular expression looks for patterns within the text. For example, you may want to verify content that may change or vary, such as an offer expiration date. Since you don’t know exactly what date the server will return, you can use a regular expression to recognize the date in the content.
Choosing your content for your checks
Okay, so you know you’re going to use a text string or a regular expression for your content check. But, what type of things should you consider when choosing the content for matching?
Use content that is unique to the page or response
Make sure the content is unique to the response. Avoid using any content from the page headers and footers. Headers and footers tend to have static text that repeats from page to page. A match in the header or footer doesn’t guarantee that the match is for the page you intended. Also, hackers may replicate your header and footer.
Don’t use dynamic or rapidly changing content
Do not use generated content unless that content is consistent and dependable or easily matched using a regular expression. For example, on an e-commerce site, using a product name may not work well since the availability of inventory frequently changes, but using a product category may work.
A caveat for multilingual sites
If you publish a multilingual site that uses the visitor’s location to determine the default display language, using visible text such as labels is not going to work for you. However, you can use underlying element attributes such as the element ID for an item container.
Setting up your content checks
Based on the monitor type, you need to consider a few things when setting up your content checks.
Uptime Monitors
Uptrends uptime/availability monitors for both webpages and web services give you the opportunity to add a free content check. It is important to note that HTTP/HTTPS monitors do not use a browser and do not process the file returned in the response. The monitor only checks the initial response body (HTML, JSON, or XML) for the match, so the content for your match must be part of the initial response. You can, in some circumstances, search the response header; contact support for assistance with response header content checks.
To add a content check to your HTTP/HTTPS uptime monitors use the Alert conditions tab in your monitor settings.
Real Browser and Full Page Check Monitors
Uptrends’ Web Performance Monitoring (Real Browser and the Full Page Check) loads your entire page into a browser. Uptrends retrieves the page’s entire contents and renders the page. You can use most any content on the page for matching provided Uptrends receives the content as an HTML (text/html) or plain text (text/plain) MIME (Multipurpose Internet Mail Extensions) type. Make sure the content you check for is in one of the eligible text file types.
To add a content check to your Real Browser and Full Page Check monitors, use the Alert conditions tab in your monitor settings.
Content checks and other monitor types
Uptrends DNS and SSL certificate monitors allow you to check specific DNS records or certificate fields. Although you can set them up to only alert you on a resolution error or certificate error, adding content checks is a fundamental part of making sure that nobody has compromised your DNS or certificate.
SSL certificates
You can compare SSL certificate fields, such as the serial number. You may also check the SSL certificates’ fingerprint. Changes in any of these fields may indicate that someone has hacked your certificate. The example in Figure 3 checks for a matching serial number.
The SSL Certificate Monitor also checks for errors and expiration dates. The monitor can warn you about impending certificate expirations based on your monitor settings. SSL Certificate monitors are also a fantastic way to keep all your certificate information in one place. Too frequently, certificates expire because the certificating company has old contact information causing renewal reminders to go to dead inboxes.
Figure 3: Matching SSL certificate values.
DNS Monitors
Besides resolve errors, a DNS monitor can check individual DNS records. The most commonly checked value is the A or AAAA record. You enter the site’s URL, and the monitor compares the returned IP address with the IP address you entered (See Figure 4).
Another common check is on the SOA (Start of Authority) record. The SOA record includes the DNS entry’s serial number. The DNS system increments this number whenever anybody makes changes to any field in the DNS entry. So, a mismatch is a good indicator that your record has changed. Learn more about setting up a DNS serial number check.
Figure 4: Checking the A record in a DNS monitor.
Self-Service Transactions or Web Application Monitors
Hopefully, your transaction monitors are full of content checks, and you’ve included a content check after every navigation or content update on a page. Content checks verify that the correct page loaded and that a user interaction successfully completed, such as switching to a new browser tab or a page submit. You can make as many content checks as you want, but please be sure to include at least one with each new step. Learn more about content checks in your Web Application Monitoring.
Multi-step API
We don’t need to go into the content match discussion for Multi-step API since content validation is central to monitoring your APIs. If you would like to learn more about content comparison and validation in your Multi-Step API calls, visit our Knowledge Base.
Conclusion
Thanks for reading. Now you are a content check pro! In review, content checks help you to
- Detect hackers
- Check for missing content
- Check for the wrong content
Content checks catch errors that your monitor can’t catch on its own such as a hijacked DNS. Most monitor types allow you to add a content check, and checks are free with your monitor. If you need help with your content checks, we have Knowledge Base articles to help you out, and you can always open a ticket for support.
