Ilya Hoilik Posted February 5, 2015 Share Posted February 5, 2015 Can you include robotstxt.txt file for IPS 4.0 to archive with RC version? Link to comment Share on other sites More sharing options...
Ilya Hoilik Posted February 19, 2015 Author Share Posted February 19, 2015 Link to comment Share on other sites More sharing options...
White Miku Posted March 29, 2015 Share Posted March 29, 2015 +1Can you add an example of robots.txt to archives? Link to comment Share on other sites More sharing options...
Ilya Hoilik Posted May 2, 2015 Author Share Posted May 2, 2015 Bump Link to comment Share on other sites More sharing options...
Management Lindy Posted May 3, 2015 Management Share Posted May 3, 2015 We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. Link to comment Share on other sites More sharing options...
pyro888 Posted June 8, 2015 Share Posted June 8, 2015 We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. Why is it relevant for Google to index for example the ?do=reportComment pages??? Link to comment Share on other sites More sharing options...
opentype Posted June 8, 2015 Share Posted June 8, 2015 Why is it relevant for Google to index for example the ?do=reportComment pages??? It isn’t, and that’s why Google won’t usually even see that link, because it gets hidden through the guest permissions – that’s what Lindy explained. Link to comment Share on other sites More sharing options...
pyro888 Posted June 8, 2015 Share Posted June 8, 2015 Hmm, in guest mode I receive indeed a 404 error (good).Google's index is not updated yet then. The pages are cached around March this year. Link to comment Share on other sites More sharing options...
Unlucky Posted May 11, 2016 Share Posted May 11, 2016 can you disallow google from indexing the members profiles (/profile/) from within admin setting guest restrictions? Thanks, Link to comment Share on other sites More sharing options...
Colonel_mortis Posted May 11, 2016 Share Posted May 11, 2016 On 03/05/2015 at 8:24 AM, Lindy said: We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. Yes there is - anything with a CSRF key in the URL will cause issues. Link to comment Share on other sites More sharing options...
recifbox Posted May 12, 2016 Share Posted May 12, 2016 on the "Use recommended settings?" i'm on "manual" settings but when we take a look at the xml generated by the sitemap.php we can see clearly that the option as little as no effect and yes we have open support incident on this and other bug on the sitemap, url issue etc etc... Link to comment Share on other sites More sharing options...
tnn Posted May 12, 2016 Share Posted May 12, 2016 3 hours ago, recifbox said: on the "Use recommended settings?" i'm on "manual" settings but when we take a look at the xml generated by the sitemap.php we can see clearly that the option as little as no effect and yes we have open support incident on this and other bug on the sitemap, url issue etc etc... I think you should have created a new topic on your sitemap issue. This topic is about robots.txt started by Ilya Hoilik. Link to comment Share on other sites More sharing options...
MeMaBlue Posted May 18, 2017 Share Posted May 18, 2017 hello, @Colonel_mortis is there a workaround on this? what issues do you mean ? can you please elaborate? @Lindy hello, is there any documentatiion specifically that will explain this to a newbie " Guest links can be removed entirely for inaccessible sections" Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.