Search engine crawlers and social sharing platforms commonly favour sites that return content rendered as static html. If your site is a SPA, PWA, or depends on JavaScript to render indexable content – you might not be visible on these platforms.
Google for example has only started crawling sites using a modern Chrome JavaScript browser, previously they were stuck crawling with an ancient Chrome 41.
Search engines require extra resources on their server to render JavaScript, your site will likely be added to a que and may not get updated in the SERPs as quickly as server side rendered page.
A common method for turning client side rendered app into a server side rendered site is to combine Headless Chrome and an Express server to create your pre-rendered HTML. This can become costly if you are processing lots of JavaScript into HTML – you may decide to detect the client connecting to your site and only serve it to Google and selected user agents.
Setting up a crawl is simple and only requires a couple of steps:
Create a html crawl and go into the advanced settings:
Set the User Agent to Use to Google Smartphone and press the Start Audit button.
There are some caveats, if the site is validating Googlebot against their published IP addresses then this method will not work.
