App Engine: How to "reset" the datastore?

Question

Well, I'm developing in App Engine (Java) and after a lot of tries and deployments, I need to reset the datastore. There is a lot of random data I added to test performance, and besides that the entities changed a lot, so I need to delete all: data, tables, indexes.

How can I do that?

score 31 · Answer 1 · answered Aug 18 '10 at 21:39

31

Sorry to wake this thread up, but just in case I'd like to add a tip for noobs like me (finally found the answer in google documentation) :

If you want to reset the Local datastore (for example while developping using eclipse) all at once, shut down the server, find the file 'local_db.bin' in your project (should be in the WEB-INF/appengine-generated/ directory), and delete it.

Works fine with java, didn't try with python yet.

++

answered Aug 18 '10 at 21:39

Kohistan

321
3
2

Intellij path: PROJECT_ROOT\out\artifacts\PROJECT_NAME_war_exploded\WEB-INF\appengine-generated\local_db.bin – unify Feb 10 '14 at 17:02
This is the correct answer for local development. Currently in the GCE, you should use the Datastore Admin which allows you to drop one or more tables. – Dominic Tracey Mar 18 '16 at 16:50

score 17 · Answer 2 · answered Aug 27 '10 at 04:50

17

Clearing the Development Server Datastore

The development web server uses a local version of the datastore for testing your application, using temporary files. The data persists as long as the temporary files exist, and the web server does not reset these files unless you ask it to do so.

If you want the development server to erase its datastore prior to starting up, use the --clear_datastore option when starting the server:

dev_appserver.py --clear_datastore helloworld/

Using the Datastore

answered Aug 27 '10 at 04:50

Tamas Kalman

1,925
1
19
24

very old thread, I know… but anyway, there's a also checkbox "Clear DataStore on Launch" in the "Application Settings" menu of the GoogleAppEngineLauncher. at least in version 1.8.8 for Mac OS X – pierroz Feb 05 '14 at 20:35
2

If you're on a Mac, you may need to use `--clear_datastore=1`. – Eric Walker Mar 31 '14 at 05:25

score 16 · Accepted Answer · edited May 23 '17 at 12:00

16

There is no built in command equivalent to DROP TABLE or TRUNCATE TABLE in SQL. You just need to create a "delete everything" page in your app, then repeatedly call that page via a script. In that page, you want to delete as many entities as you can yet still reasonably expect to finish before the request times out. The exact code depends on whether you're using JDO/JPA or the low level API. (the low level API will be faster because you can use batch operations.)

This previous SO question is pretty much the same, only for Python

edited May 23 '17 at 12:00

Community

1
1

answered Jul 20 '09 at 17:42

Peter Recore

14,037
4
42
62

I'm using JPA. Will this delete indexes too? – Damian Jul 20 '09 at 18:28
2

The indexes need to get updated every time you delete something, otherwise queries that use indexes would be returning bad data. So when you delete entity X, all index entries pointing to X should be deleted as well. Eventually you'll have empty indexes when you've deleted all your entities. – Peter Recore Jul 20 '09 at 19:38
5

Deleting indexes using appcfg first is a good idea - it'll make the delete process faster and cause it to consume less CPU updating indexes that you're just going to delete anyway,. – Nick Johnson Jul 22 '09 at 14:02

score 9 · Answer 4 · edited Aug 13 '11 at 02:29

Just execute a query without a filter to fetch all the entities, and delete them one by one.

import javax.servlet.http.*;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;

public class MyServlet extends HttpServlet {
        public void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws IOException {

        DatastoreService datastore = 
                    DatastoreServiceFactory.getDatastoreService();

    Query mydeleteq = new Query();
    PreparedQuery pq = datastore.prepare(mydeleteq);
    for (Entity result : pq.asIterable()) {
        datastore.delete(result.getKey());      
    }   
}

This works great when embedding the datastore locally without the overhead of really starting a server — Dr. Max Völkel, Nov 22 '11 at 17:03
Put datastore.delete(result.getKey()); in a try/catch(IllegalArgumentException e) {} to avoid Key names beginning and ending with __ which are reserved and can't be deleted — M.ES, Jul 14 '12 at 14:46

score 4 · Answer 5 · answered Sep 07 '09 at 22:25

4

sorry to be so late on this, but I was just trying to do the same myself...

I logged into my account (appengine.google.com) and found the option to browse the datastore through an admin utility (datastore/dataviewer)... that allows create/update/delete.

answered Sep 07 '09 at 22:25

DanDan

101
1
4

9

If you tweak the limit=20 in the URL you can up the limit. In my case increasing to 100 let me delete a chunk of data in a couple requests (and saved me writing a tool/page to do this). – Michael Twomey Sep 15 '09 at 10:07

jonmiddleton · Answer 6 · 2010-11-30T22:52:25.690

2

Delete all (or a part) of your application’s data is now part of the Admin console

To enable this functionality, simply enable the following builtin in your app.yaml file:

builtins:
- datastore_admin: on

Adding these lines to app.yaml enables the “Datastore Admin” page in your app’s Admin Console

edited Nov 30 '10 at 22:52

answered Jan 24 '10 at 21:49

jonmiddleton

1,122
14
16

score 2 · Answer 7 · answered Apr 20 '10 at 18:26

According to GAE docs you can delete multiple objects in JDO, call the PersistenceManager's deletePersistentAll(...) method with a Collection of objects.

PersistenceManager pm = PMF.get().getPersistenceManager();

Query query = pm.newQuery("select from " + Your.class);

List<Your> objs = (List<Your>) query.execute();

pm.deletePersistentAll(objs);

score 1 · Answer 8 · edited Oct 01 '14 at 11:19

1

Out of context for java dev but since there's few documentation here's how to do it in go :

keys, _ := datastore.NewQuery("").KeysOnly().GetAll(c, nil)
datastore.DeleteMulti(c, keys)

edited Oct 01 '14 at 11:19

Xan

74,770
16
179
206

answered Oct 01 '14 at 11:02

Solido

31
4

score 1 · Answer 9 · answered Jun 02 '15 at 14:11

1

If you use maven in your project, you can just do a "mvn clean install". That will reset the datastore locally of course.

answered Jun 02 '15 at 14:11

Dennis De Reyer

196
7

You're right, and its a pain. Each time I want a clean build, the DB gets cleared. I want new code, I don't want to recreate the Data each time! – Teddy Nov 13 '22 at 16:32

score 1 · Answer 10 · answered Jul 05 '16 at 17:48

1

Deleting local data can be done by opening http://localhost:8000/datastore

answered Jul 05 '16 at 17:48

Yash Sharma

811
1
9
27

score 0 · Answer 11 · answered Jan 16 '13 at 21:16

0

I was using app engine with Google Coursebuilder and had to use this command to clear the datastore:

python dev_appserver.py --clear_datastore /path/to/app

answered Jan 16 '13 at 21:16

cwd

53,018
53
161
198

score 0 · Answer 12 · answered Jan 12 '11 at 06:20

0

in my case (working with eclipse plugin and play framework) I had to stop the application, delete file /apps/crud-gae/tmp/datastore and then restart the app

it worked for me... working locally, of course...

answered Jan 12 '11 at 06:20

opensas

60,462
79
252
386

score 0 · Answer 13 · answered Feb 15 '18 at 00:31

To add another bit of helpful info: If using Eclipse and you want to clear the local datastore, look for this console msg:

INFO: Local Datastore initialized: 
Storage: C:\Users\eric\workspace\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\appname\WEB-INF\appengine-generated\local_db.bin

It only shows up after you do something to force datastore initialization, e.g. which could be refreshing the entity list on the admin page. Then stop your server and delete the file, and when you restart you should see:

INFO: The backing store, C:\Users\eric\workspace\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\appname\WEB-INF\appengine-generated\local_db.bin, does not exist. It will be created.

score 0 · Answer 14 · answered Jan 14 '12 at 13:05

0

When working locally, on Windows 7 the file is user\UserName\AppData\Local\Temp\dev_appserver.datastore

answered Jan 14 '12 at 13:05

user1055761

1,071
1
12
28

App Engine: How to "reset" the datastore?

14 Answers14

Linked