User-Agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php # Removing excess pages which are noindexed and provide no value to Google Disallow: /category* Disallow: /tag/* Disallow: /author/* Disallow: /email Disallow: /lp/* Disallow: /ss-* Disallow: /yt/* Disallow: /20* Disallow: */embed/ Disallow: */feed/ Disallow: /privacy-policy/ Disallow: /members/ Disallow: /cg* # Removing some random URL parameters Disallow: *lang=* Disallow: *orderby=* Disallow: *add-to-cart=* Disallow: *p=* Disallow: *post_type=* # Allow several posts with year in slug Allow: /2017-* Sitemap: https://simpleprogrammer.com/sitemap_index.xml Sitemap: https://simpleprogrammer.com/HTTP_Sitemap_SP.xml Sitemap: https://simpleprogrammer.com/store/sitemap_index.xml