Robots and Sitemap
In this lesson, you will add even more for SEO. Your application deserves robots.txt and proper sitemap.
SEO#
To make your application more accessible for search engines you should provide them with the robots.txt and sitemap.xml files
Robots.txt#
Start by informing robots what they should and shouldn't traverse on your website. This helps avoid unnecessary traffic from them. Create a new file, src/robots.txt, and feed it with data:
User-agent: *
Disallow: /favorites/
Disallow: /login/
Disallow: /assets/
Disallow: /api/
Sitemap: https://my-universal-app.herokuapp.com/sitemap.xml
This configuration will prevent search engine robots from visiting paths that they have no need to traverse:
/favorites is only available to authenticated users.
/login doesn't contain any content that is critical from the SEO perspective.
/assets is where static files reside (this doesn't mean that Google won't index images).
/api is where the backend API resides.
The last line is to inform robots where they should look for the sitemap of your application.
This page is a preview of The newline Guide to Angular Universal