Beacon generates robots.txt per site
#700
Conversation
make it backwards compatible, ie: works even with a single Endpoint (the default setup)
| @@ -0,0 +1,3 @@ | |||
| # http://www.robotstxt.org | |||
There was a problem hiding this comment.
We could disallow the admin prefix here as well. See https://github.com/BeaconCMS/beacon_live_admin/blob/f8ff1621253c30c5f143655c11acf1116fa1c1e4/lib/beacon/live_admin/router.ex#L88
We need to use scoped_path to build the full path including the scope path. So if that attribute is present in the router we could include it as Disallow: <%= @admin_prefix %> otherwise just ignore the line since the admin might not even be included in the router.
And we'll need at least a simple test because that function might change and we need a failing test if that happens.
There was a problem hiding this comment.
We could disallow the admin prefix here as well
I don't think that will be included. This PR only uses the beacon_site macro for generation
Resolves #219
Depends on #694, #687