RE: Robots.txt impacting https://templates.openoffice.org/ on Bing

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

RE: Robots.txt impacting https://templates.openoffice.org/ on Bing

Bing Webmaster Outreach

[hidden email] [hidden email] [hidden email]

 

Hi,

 

Adding additional recipient to this email. Request your kind attention on below email.

 

Note: In case you are not the relevant person to receive this email, kindly forward it to the relevant person in your company. Relevant person would be Website admin, Webmaster, CTO, SEO Manager, Tech support, etc.

 

Thanks & Regards,                                                                                                                                                 

Kanhu Charana Badatia

Bing Webmaster Outreach Co-Ordinator

https://www.bing.com/toolbox

Privacy Policy

 

From: Bing Webmaster Outreach
Sent: Thursday, May 23, 2019 5:06 PM
To: [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Robots.txt impacting https://templates.openoffice.org/ on Bing

 

Hello,

 

I am contacting you from Microsoft on behalf of the Bing Search engine (https://www.bing.com/) in regards to your robots.txt files: https://templates.openoffice.org/

 

When we trying to find your https://templates.openoffice.org/sitemap.xml , It is showing Page Not Found on your website. So, could you please update your sitemap with lastmod and submit it in Bing Webmasters. By providing the last modification timestamp, you enable search engine crawlers to retrieve only a subset of the Sitemaps in the index i.e. a crawler may only retrieve Sitemaps that were modified since a certain date. This incremental Sitemap fetching mechanism allows for the rapid discovery of new URLs on very large sites.

 

Our customers have alerted us that your website was partially absent from our search results. We have discovered that you have a crawl delay directive in your robots.txt which is impacting the freshness of your content on Bing Search Engine.

 

Major search engines like Google and Yandex don’t follow the crawl delay rules while Bing does. We would be pleased if you could edit your robots.txt file to include a bingbot specific rule with removed or reduced crawl delay, which will in turn increase traffic to your site via our search results.

 

Existing Robots.txt

Modify to

 

User-agent: *

Crawl-delay: 10

# CSS, JS, Images

Allow: /misc/*.css$

Allow: /misc/*.css?

Allow: /misc/*.js$

Allow: /misc/*.js?

Allow: /misc/*.gif

Allow: /misc/*.jpg

Allow: /misc/*.jpeg

Allow: /misc/*.png

Allow: /modules/*.css$

Allow: /modules/*.css?

Allow: /modules/*.js$

Allow: /modules/*.js?

Allow: /modules/*.gif

Allow: /modules/*.jpg

Allow: /modules/*.jpeg

Allow: /modules/*.png

Allow: /profiles/*.css$

 

 

 

User-agent: *

Crawl-delay: 3

# CSS, JS, Images

Allow: /misc/*.css$

Allow: /misc/*.css?

Allow: /misc/*.js$

Allow: /misc/*.js?

Allow: /misc/*.gif

Allow: /misc/*.jpg

Allow: /misc/*.jpeg

Allow: /misc/*.png

Allow: /modules/*.css$

Allow: /modules/*.css?

Allow: /modules/*.js$

Allow: /modules/*.js?

Allow: /modules/*.gif

Allow: /modules/*.jpg

Allow: /modules/*.jpeg

 

 

In case you don’t want to modify the robots.txt files, please let us know if we can bypass the rule and get your pages indexed properly.

 

I also invite you to register your site on Bing Webmaster Tools where you can configure your own settings, including hourly crawl control. You can use the Crawl Control feature on our Bing Webmaster Tools, where you can configure hourly crawl control based on the peak usage time for your site and obtain the best results for your site.

 

Please let me know should you have any question.

 

Note: In case you are not the relevant person to receive this email, kindly forward it to the relevant person in your company. A relevant person would be Website admin, Webmaster, CTO, SEO Manager, Tech support, etc...

 

Thanks & Regards,

Appaji V Sarakanam

Bing Webmaster Outreach Co-ordinator

www.bing.com/webmaster

Privacy Policy