Angular2 + webpack do not deploy robots.txt

Question

I am creating a web site with Angular2@2.1.2. I am using Webpack with default settings (as a dependency).

Here is my package.json

"dependencies": {
"@angular/common": "2.1.2",
"@angular/compiler": "2.1.2",
"@angular/core": "2.1.2",
"@angular/forms": "2.1.2",
"@angular/http": "2.1.2",
"@angular/platform-browser": "2.1.2",
"@angular/platform-browser-dynamic": "2.1.2",
"@angular/platform-server": "2.1.2",
"@angular/router": "3.1.2",
"@ngrx/core": "1.2.0",
"@ngrx/effects": "2.0.0",
"@ngrx/store": "2.2.1",
"angular2-toaster": "^1.0.1",
"awesome-typescript-loader": "2.2.1",
"bootstrap": "3.3.7",
"bootstrap-select": "1.11.2",
"eonasdan-bootstrap-datetimepicker": "4.17.42",
"es5-shim": "4.5.9",
"intl": "1.2.5",
"jquery": "3.1.0",
"moment": "2.15.1",
"ng2-modal": "0.0.21",
"ng2-pagination": "^0.4.1",
"ngrx-store-logger": "^0.1.7",
"npm": "3.9.3",
"reflect-metadata": "0.1.8",
"rxjs": "5.0.0-beta.12",
"ts-helpers": "1.1.1",
"zone.js": "0.6.25"
},
"devDependencies": {
"@types/jasmine": "2.2.34",
"angular-cli": "1.0.0-beta.19-3",
"codelyzer": "~0.0.26",
"core-js": "2.4.1",
"jasmine-core": "2.5.0",
"jasmine-spec-reporter": "2.7.0",
"karma": "1.3.0",
"karma-chrome-launcher": "2.0.0",
"karma-coverage": "1.1.1",
"karma-jasmine": "1.0.2",
"karma-phantomjs-launcher": "1.0.2",
"karma-remap-istanbul": "0.2.1",
"karma-verbose-reporter": "0.0.3",
"node-sass": "3.10.3",
"protractor": "4.0.5",
"ts-node": "1.3.0",
"tslint": "3.15.1",
"typescript": "2.0.2"
}

I have added a robots.txt file in the assets/ directory. I was thinking that the builder (npm build) recognizes this file and put it at the root of the application, but it does not, it is still in the assets directory.

Do I miss something ?

Thanks.

Guymage · Accepted Answer · 2017-05-01T02:48:53.527

41

I have found my solution in this issue: https://github.com/angular/angular-cli/issues/1942

robots.txt is in src/ directory
Modify angular-cli.json

"apps": [
{
  "root": "src",
  "outDir": "dist",
  "assets": ["assets", "robots.txt"],
 ....

Use the assets array to declare files you want to be placed in the root of dist/

edited May 01 '17 at 02:48

answered Feb 10 '17 at 05:52

Guymage

1,524
1
14
21

score 21 · Answer 2 · edited Aug 12 '21 at 13:24

21

I made this work with Angular6.

Place your robots.txt in the src folder, the same folder as the favicon.ico.

In your angular.json file

              "assets": [
              "src/favicon.ico",
              "src/assets",
              "src/robots.txt"
            ],

You may need to ng serve again, then point to http://localhost:4200/robots.txt

edited Aug 12 '21 at 13:24

Fabian

5
3

answered Feb 28 '19 at 21:27

Jason

931
2
12
26

If I am caching all unknown routes (path:'**') what can I do? should I add a route for the robots.txt? – Raikish Oct 22 '20 at 11:25
2

I just tried to one of my applications and seems to be working on angular 10.1.1 using angular universal server side rendering. it hasn't problems with (path:'**'), I recommend update your answer removing the "This will only work if you are not catching all unknown routes (path:'**') in your router module(s). " sentence. – Raikish Oct 22 '20 at 16:12
this did not work for me, instead glob worked as mentioned here https://stackoverflow.com/questions/61902703/ignore-a-path-in-the-root-routermodule – Naga Jul 24 '22 at 18:49

score 1 · Answer 3 · answered Sep 24 '20 at 20:03

I wanted my robots.txt to be in my src/assets folder rather than just src. I found that the following being added to my angular.json in the assets array did the trick:

"assets": [
  "src/favicon.ico",
  "src/assets",
  {
    "glob": "robots.txt",
    "input": "src/assets",
    "output": "./"
  }
],

Works in Angular 10 using Angular Universal/SSR.

Angular2 + webpack do not deploy robots.txt

3 Answers3

Linked