Creating a robots.txt file

Discussions

Bradley Blinn

For those that own a hoop.la site, I might make the recommendation that you research how to add a robots.txt file to your site. I ran a link checker on my SocialStrata website and I noticed there are a lot of links that would send a search engine spider down a rabbits hole. What you want is Google and Bing indexing your quality content pages. I created the following robots.txt file:

User-agent: *
Disallow: /pin/
Disallow: /display-forum-avatar/
Disallow: /join/
Disallow: /login/
Disallow: /printer-friendly-topic/
Disallow: /redirect-to-user-social-network/
Disallow: /static/

To the SocialStrata admins, I might even consider making this a default setting in everyone's account because this would improve spidering of pages and lessen the load on your server hardware.

Thoughts?

Brian LenzSocial Strata CTO

Thanks for the tip, Bradley! We likely won't exclude any pages by default, but other site admins may find it useful

1/30/159:24 AM

Add Reply

<ul id="{%=id%}ButtonGroup" class="button-group h-popup-buttons even-button-group even-1">
    <li id="{%=id%}Okay_buttonContainer"><a id="{%=id%}Okay" href="javascript:void(0);" class="button success no-margin-bottom h-alert-modal-ok-button" role="button">OK</a><script type="text/javascript">
        (function() {
            function {%=id%}Okay_click(event, data) {
            {%=id%}Popup.hide();
        }
        
                $('#{%=id%}Okay').on('click', {%=id%}Okay_click);
        })();
    </script></li></ul><a href="javascript:void(0);" class="close-reveal-modal" aria-label="Close Modal">&#215;</a><div class="last-modal-focus focus-utility-element" tabindex="0"></div>
</div>
<script type="text/javascript">
        (function() {
            {%=id%}Popup = new SS.Popup.ModalPopup('{%=id%}');
        })();
    </script>

Skip to main content

Discussions

Add Reply

Membership Required

Remove From Your Block List

Block

Please wait...