«Stores» → «Configuration» → «General» → «Design»→ «Search Engine Robots»:
After saving the configuration the robots.txt
file will be generated in the Magento 2 root folder.
«Stores» → «Configuration» → «General» → «Design»→ «Search Engine Robots»:
After saving the configuration the robots.txt
file will be generated in the Magento 2 root folder.
All great just would like to do one more addition in the post, Custom Instructions like:
Disallow: /lib/
Disallow: /*.php$
Disallow: /pkginfo/
Disallow: /report/
Disallow: /var/
Disallow: /catalog/
Disallow: /customer/
Disallow: /sendfriend/
Disallow: /review/
Disallow: /*SID=