16

I'm using the Go SDK for the Google App Engine (it uses some parts of the Python SDK and calls into dev_appserver.py when running goapp serve). By default, it seems to watch all files and folders for changes. I have a crapload of bower dependencies in the static folder, and the SDK complains that it can't watch that many files.

How to set the dev server to ignore changes in a folder?

Edit. Reading devappserver2/watcher_common.py, it seems that it ignores directories starting with .. I suppose in the worst case scenario, I could prefix folders with ., but it's a hack. There should be a configuration option, and I can't seem to find it.

Nelo Mitranim
  • 823
  • 1
  • 13
  • 13

3 Answers3

3

Using:

Google Cloud SDK 197.0.0
app-engine-python 1.9.68
app-engine-python-extras 1.9.63
bq 2.0.31
core 2018.04.06
gsutil 4.30

From the App Engine Docs, you can pass an argument, --watcher_ignore_re, to dev_appserver.py when running from the command line.

Example

dev_appserver.py app.yaml --watcher_ignore_re="(.*\.data|.*\.vscode)"

This will ignore changes to the .data and .vscode directories, relative to the directory from which the command is run (CWD).

Edit 7th June 2018.

Using:

Google Cloud SDK 204.0.0
app-engine-python 1.9.70
app-engine-python-extras 1.9.69
bq 2.0.34
core 2018.06.04
gsutil 4.31

The watcher_ignore_re seems to not be working anymore.

Rather, what worked for me is to add a skip_files directive in the app.yaml file as stated here in the docs. The dev_appserver.py command will respect the definitions in this section.

Example

Inside app.yaml of the project:

...
...
skip_files:
# default from GAE
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$

# custom entries

# dev-related files
- ptvsd
- pydev_startup.py
- .pylintrc
- .data
- .vscode

# version control files
- .git
- .gitignore
- .gitattributes

# non-application files
- README.md
- ^(.*/)?.*\.mwb(.bak)?$ # workbench models

Now, run project with command:

dev_appserver.py app.yaml
Prince Odame
  • 534
  • 7
  • 17
  • For those who faced with not implemented support of `watcher_ignore_re` and `skip_files_re` on Windows [my answer](https://stackoverflow.com/questions/50721890/google-app-engine-dev-appserver-py-watcher-ignore-re-flag-is-not-json-serializ/54155850#54155850) with workaround with implementations its support for Windows may be useful – Pavlo Zhukov Jan 12 '19 at 10:01
2

You are correct that there is currently no command-line or configuration option to specify files/directories to be ignored by the watcher.

. (dot) hack

As you've discovered, files and directories beginning with . are ignored. This is a fairly standard convention for many *nix systems. It is not really scalable however with python projects with many local dependencies.

Modifying the watcher file

You can modify this file but this too is unfavorable as it makes for very brittle changes and is prone to breaking if updates overwrite the file.

Barring the above workarounds, there's no way to achieve this. I've also not seen any mention of this being implemented in the App Engine release notes. There is however an open feature request on the public issue tracker to have the watcher ignore files specified by the skip_files yaml directive. Feel free to star that issue to get updates regarding its progress.

EDIT: Jan 4, 2017

NPM 3

For users of NPM encountering this limitation, you may find it helpful to use NPM v3 as it resolves dependencies differently.

npm3 attempts this [mitigate deep trees and redundancy] by installing some secondary dependencies (dependencies of dependencies) in a flat way, in the same directory as the primary dependency that requires it.

This can be very effective for Node users encountering the file watching limitation.

Frankenserver

Khan Academy has developed a fork of the App Engine devserver called Frankenserver. While I've not explored this option myself, there seems to be a fair amount of support and recent updates to it. From it's readme:

frankenserver's biggest advantage over the vanilla SDK is in how it watches the files in your app for changes. It does this much more efficiently by 1) using a native FSEvents-based file watcher on Mac OS X and 2) respecting the skip_files directive in your app.yaml.

Though this is not an official Google solution, it may be a worthwhile workaround for time being if this limitation affects you direly.

Nicholas
  • 1,676
  • 12
  • 35
  • starred the feature request, unfortunately it's been starred for 3 years with acknowledge – user7180 Jan 04 '17 at 14:45
  • 1
    Thanks for bringing this up. I'll investigate [`Issue 9300`](https://code.google.com/p/googleappengine/issues/detail?id=9300) to see what's happening with this internally. I'll also update my answer with some additional workarounds that may be helpful in the meantime. – Nicholas Jan 04 '17 at 15:18
0

I found that the comment37 in the Issue 9300 works for me.

In case, the link is not accessible anymore. here is the comment37:

In theory this code exists now, but there is a bug. It does use the skip_files directive, but instead of matching against relative paths, it uses absolute paths.

So if I have:

skip_files:

- ^node_modules(/.*)?

It will attempt to match /Users/myuser/Projects/myproject/node_modules/ and fail.

Adding an additional entry should fix it and let it skip your node_modules directory:

skip_files:

- ^node_modules(/.*)?

- .*node_modules

This is the hack-workaround I'm using now, which at least allows me the convenience of not needing to go in and edit the gcloud files directly each time they re-release!

highlight of the most important part:

skip_files: 
- ^node_modules(/.*)? 
- .*node_modules
Edward Fung
  • 426
  • 8
  • 16