AngularJS SEO with Prerender.io | DigitalOcean

Gods time Nwachukwu July 13, 2024

Introduction

AngularJS is an excellent framework for building websites and apps. Built-in routing, data-binding, and directives among other features enable AngularJS to completely handle the front-end of any type of application.

The one pitfall to using AngularJS (for now) is Search Engine Optimization (SEO). In this tutorial, we will go over how to make your AngularJS website or application crawlable by Google.

Search engines crawlers (or bots) were originally designed to crawl the HTML content of web pages. As the web evolved, so did the technologies powering websites and JavaScript became the de facto language of the web. AJAX allowed for asynchronous operations on the web. AngularJS fully embraces the asynchronous model and this is what creates problems for Google’s crawlers.

If you are fully utilizing AngularJS, there is a strong possibility that you will only have one real HTML page that will be fed HTML partial views asynchronously. All the routing and application logic is done on the client-side, so whether you’re changing pages, posting comments, or performing other CRUD operations, you are doing it all from one page.

Rest assured, Google does have a way of indexing AJAX applications, and your AngularJS app can be crawled, indexed, and will appear in search results just like any other website. There are a few caveats and extra steps that you will need to perform, but these methods are fully supported by Google. To read more about Google’s guidelines for crawlable AJAX content visit Google’s Webmaster AJAX Crawling Guidelines.

Our application will be able to be rendered by Google bot and all his friends (Bing bot). This way, we won’t run into the problem shown in the picture above. We’ll get nice search results as our users expect from us.

When a search engine crawler visits your app and sees the it will add an ?_escaped_fragment_= tag to your URL.
Your server will intercept this request and send it to the middleware that will handle the special crawler request. For this article, we have chosen Prerender.io so the next step is specific to Prerender.io.
Prerender.io will check to see if the requested page has an existing snapshot (or cached page), if it does, it will serve that page to the crawler, if it does not, Prerender will make a call to PhantomJS which will render the page in its entirety and show it to the crawler.
Non-cached pages that require the call to PhantomJS will take longer to render leading to a much longer response time, so it’s a good idea to cache pages often.
There are additional ways to do this!

Alternatives:

Set up your own Prerender service using Prerender.io open-source code
Use a different existing service such as BromBone, Seo.js or SEO4AJAX
Create your own service for rendering and serving snapshots to search engines

Prerender.io is a service that is compatible across a variety of different platforms including Node, PHP, and Ruby. The service is fully open-source but they do offer a hosted solution if you do not want to go through the hassle of setting up your own server for SEO. The folks over at Prerender believe that SEO is a right, not a privilege and they have done some great work extending their solution, adding a lot of customizable features and plugins.

We will be building a simple Node/AngularJS application that has multiple pages with dynamic content flowing throughout. We will use Node.js as our backend server with Express. Check out the Node package.json file below to see all of our dependencies for this tutorial. Once you are ready, sign up for a free prerender.io account and get your token.

{ “name”: “Angular-SEO-Prerender”, “description”: “…”, “version”: “0.0.1”, “private”: “true”, “dependencies”: { “express”: “latest”, “prerender-node”: “latest” } }

Now that we have our package.json ready to go, let’s install our Node dependencies using npm install.

The setup here is pretty standard. In our server.js file we will require the Prerender service and connect to it using our prerender token.

var express = require(‘express’); var app = module.exports = express(); app.configure(function(){ app.use(require(‘prerender-node’).set(‘prerenderToken’, ‘YOUR-TOKEN-HERE’)); app.use(express.static(“public”)); app.use(app.router); }); app.get(‘*’, function(req, res){ res.sendfile(‘./public/index.html’); }); app.listen(8081); console.log(“Go Prerender Go!”);

The main page is also pretty standard. Write your code like you normally would. The big change here will simply be adding to the of your page. This meta tag will tell search engine crawlers that this is a website that has dynamic JavaScript content that needs to be crawled.

Additionally, if your page is not caching properly or it’s missing content you can add the following script snippet: window.prerenderReady = false; which will tell the Prerender service to wait until your entire page is fully rendered before taking a snapshot. You will need to set window.prerenderReady = true once you’re sure your content has completed loading. There is a high probability that you will not need to include this snippet, but the option is there if you need it.

That’s it! Please see the code below for additional comments.

Scotch Tutorial | {{ seo.pageTitle }}

Welcome to the Angular SEO Prerender Tutorial

In our app.js, the page where we define our AngularJS code, we will need to add this code to our routes config: $locationProvider.hashPrefix(‘!’);. This method will change the way your URLs are written.

If you are using html5Mode you won’t see any difference, otherwise, your URLs will look like http://localhost:3000/#!/home compared to the standard http://localhost:3000/#/home.

This #! in your URL is very important, as it is what will alert crawlers that your app has AJAX content and that it should do its AJAX crawling magic.

In the above code, you can see how we handle Angular routing and our different pageTitle and pageDescription for the pages. These will be rendered to crawlers for an SEO-ready page!

When a crawler visits your page at http://localhost:3000/#!/home, the URL will be converted to http://localhost:3000/?escaped_fragment=/home, once the Prerender middleware sees this type of URL, it will make a call to the Prerender service. Alternatively, if you are using HTML5mode, when a crawler visits your page at http://localhost:3000/home, the URL will be converted to http://localhost:3000/home/?escaped_fragment=.

The Prerender service will check and see if it has a snapshot or already rendered page for that URL, if it does, it will send it to the crawler, if it does not, it will render a snapshot on the fly and send the rendered HTML to the crawler for correct indexing.

Prerender provides a dashboard for you to see the different pages that have been rendered and crawled by bots. This is a great tool to see how your SEO pages are working.

I recently got a chance to chat with the creator of Prerender.io and asked him for some tips on getting your single-page app indexed. This is what he had to say:

Serve the crawlers prerendered HTML, not JavaScript,
Don’t send soft 404s
If you’re sticking with #s for your URLs, make sure to set the hashPrefix(‘!’) so that the URLs are rewritten as #!s
If you have a lot of pages and content, be sure to include a sitemap.xml and robots.txt
Google crawls only a certain number of pages per day, which is dependent on your PageRank. Including a sitemap.xml will allow you to prioritize which pages get indexed.
When testing to see how your AngularJS pages render in Google Webmaster Tools, be sure to add the #! or ?escaped_fragment= in the right place as the manual tools do not behave exactly as the actual crawlers do.

Hopefully, you won’t let the SEO drawback of Angular applications hold you back from using the great tool. There are services out there like Prerender and ways to crawl AJAX content. Make sure to look at the Google Webmaster AJAX Crawling Guidelines and have fun building your SEO-friendly Angular applications!