1

I'm after some advice. I am working on a project using Entity Framework as the Orm. We are using a code-first approach throughout. We are using behaviour driven development too and have created a set of automated web tests using specflow and selenium. I need to be able to delete all data from the database that was created during a test. So ideally, in the test hook that I've created that gets executed after a test, I want to delete all data that was added during the test.

Ideally I'd like to keep with code first approach but I'm open to suggestions. I'd like to see how others have provided a solution and get some advice from them.

Nore Gabbidon
  • 351
  • 1
  • 4
  • 10

4 Answers4

1

Not specific SpecFlow or any other test framework, but I would highly recommend using in-memory database.

https://www.nuget.org/packages/Effort.EF6/

In combination with DI contaner you get very good control of database lifetime in tests.

LocalIocManager.IocContainer.Register(
    Component.For<DbConnection>()
    .UsingFactoryMethod(Effort.DbConnectionFactory.CreateTransient)
    .LifestyleCustom<ManualLifestyleManager>());

You can reuse same Seeders to populate test data, dump or restore from CSV etc.

Edgars Pivovarenoks
  • 1,526
  • 1
  • 17
  • 31
  • Now this is definitely something I would consider for the future. It isn't suitable for my current project but definitely something I would want to use. Thanks for the comment. Much appreciated. – Nore Gabbidon Nov 29 '16 at 10:28
0

If BDD, then I believe you are using Specflow. Specflow provides hooks where you can place your tear down methods. To collect the data you created during test execution you can store it in the ScenarionContext provided by also Specflow. In the teardown you can access it and read out your data and let EF delete them.

I believe you have 3 options:

1, You have a database with minimal dataset and every test have its own test data in the Background section. When background step puts data into db you can store this data in your ScenarioContext and in the AfterScenario hook you can delete these records. It is good if you satisfied with the ownership over your test data. This way you might have other created data which is not under your control.

2, The solution for the 1, is that in AfterScenario or in BeforeScenario you run a script which cleans up and inserts a certain dataset into database. The disadvantage of this solution is that, the delete/truncate time and populating time might be too much for you.

3, In the AfterScenario or BeforeScenario, or periodically during test execution you can restore your database. I believe this way depends on how much time is needed to restore your db.

I believe you can combine the options above as it is feasible for you. Your solution depends on the database type, the data you work with and the environments you have.

AndrasCsanyi
  • 3,943
  • 8
  • 45
  • 77
  • Yes, I'm using this approach already, and there are already occasions where I'm managing data that has been managed in the tests. However, as a result of the automated web testing, there will also be data that is inserted/amended during a test that cannot be controlled. What I'm looking for is a solution to 'reset' the database back to the state it was before the test was run. – Nore Gabbidon Nov 27 '16 at 14:53
  • Why not truncating the tables and insert the initial data into it? It is part of test data management. You have to have control over the test data. Store it in a sql script file and run it at every test or when it is needed. – AndrasCsanyi Nov 27 '16 at 16:31
  • I've implemented this by doing the following - Added an EF initializer that is specific to my test suite only. Its based on CreateDatabaseIfNotExists initializer, so if the database doesn't exist it will be created. If it does exist, then I've enabled so that all entities are retrieved and truncated, and then the seeding runs again. I've played with leaving it to just delete and create the database, which works fine. The issue is that as the tests start to grow and data gets more expansive, the performance of the initializer when dropping and recreating the database will be extremely poor. – Nore Gabbidon Nov 29 '16 at 10:56
  • That's why I wrote that you need to find what is feasible for you. There are no clear answer. We have all our test related test data in the particular feature's background section. At start it is inserted ( 1 sec ), at tear down it is deleted (1 sec) (MSSQL 2014 and a single test requires 100-200 recrods), if needed further data initialization can be used. – AndrasCsanyi Nov 29 '16 at 11:04
  • Yes I agree. I wanted to find out how others had implemented this solution. There's some great options out there that I've evaluated. My one remaining objective now is to implement the best way of truncating tables to allow a proper reseed. Thanks for the information. – Nore Gabbidon Nov 29 '16 at 11:41
0

You could take a backup of the DB in the BeforeScenario hook, and then restore that backup in the AfterScenario hook.

Sam Holder
  • 32,535
  • 13
  • 101
  • 181
0

Eventually after investigation and a bit of playing around, I achieved this by doing the following:

Created an Entity Framework Initializer based on CreateDatabaseIfNotExists (http://www.entityframeworktutorial.net/code-first/database-initialization-strategy-in-code-first.aspx):

public class DbTestInitializer : CreateDatabaseIfNotExists<DBContext>
{
    public override void InitializeDatabase(DBContext context)
    {
        if(context.Database.Exists())
        {
            // We have a database already, so we can clean entities and run seed.
            CleanseTables(context);
            Seed(context);
        }

        base.InitializeDatabase(context);
    }

    private void CleanseTables(DBContext context)
    {
        // Run the database teardown script 
        context.Database.ExecuteSqlCommand(Properties.Resources.DatabaseTeardown);
    }

    protected override void Seed(DBContext context)
    {
        this.ApplySeed(context);
    }
}

Inside the initializer InitializeDatabase method, I check to see if the database exists - if so then I execute a cleanse script that does the following:

  • Collected all foreign key references and constructed a drop constraint and create constraint script for each one. I then executed the drop constraints, truncated every table in the database (excluding the MigrationHistory table specific to EF Migrations) and reenabled the constraints using the create constraint scripts. This is done by executing the script via context.Database.ExecuteSQLCommand([script])

Here's how that script looks.

DECLARE @ConstraintsTable TABLE
(

ID INT IDENTITY(1,1),

DropConstraintScript VARCHAR(MAX),

EnableConstraintScript VARCHAR(MAX)

)

INSERT INTO @ConstraintsTable
SELECT 
'ALTER TABLE [' + ForeignKeys.ForeignTableSchema
            + '].[' + ForeignKeys.ForeignTableName + '] DROP CONSTRAINT     ['
            + ForeignKeys.ForeignKeyName + ']; ',
    'ALTER TABLE [' + ForeignKeys.ForeignTableSchema
            + '].[' + ForeignKeys.ForeignTableName
            + '] WITH CHECK ADD CONSTRAINT [' + ForeignKeys.ForeignKeyName
            + '] FOREIGN KEY([' + ForeignKeys.ForeignTableColumn
            + ']) REFERENCES [' + SCHEMA_NAME(sys.objects.schema_id)
            + '].[' + sys.objects.[name] + ']([' + sys.columns.[name]
            + ']);'
    FROM    sys.objects
    INNER JOIN sys.columns
            ON ( sys.columns.[object_id] = sys.objects.[object_id] )
    INNER JOIN ( SELECT sys.foreign_keys.[name] AS ForeignKeyName
                        ,SCHEMA_NAME(sys.objects.schema_id) AS ForeignTableSchema
                        ,sys.objects.[name] AS ForeignTableName
                        ,sys.columns.[name] AS ForeignTableColumn
                        ,sys.foreign_keys.referenced_object_id AS referenced_object_id
                        ,sys.foreign_key_columns.referenced_column_id AS referenced_column_id
                    FROM   sys.foreign_keys
                    INNER JOIN sys.foreign_key_columns
                        ON ( sys.foreign_key_columns.constraint_object_id = sys.foreign_keys.[object_id] )
                    INNER JOIN sys.objects
                        ON ( sys.objects.[object_id] = sys.foreign_keys.parent_object_id )
                    INNER JOIN sys.columns
                        ON ( sys.columns.[object_id] = sys.objects.[object_id] )
                            AND ( sys.columns.column_id = sys.foreign_key_columns.parent_column_id )
                ) ForeignKeys
            ON ( ForeignKeys.referenced_object_id = sys.objects.[object_id] )
                AND ( ForeignKeys.referenced_column_id = sys.columns.column_id )
    WHERE   ( sys.objects.[type] = 'U' )
    AND ( sys.objects.[name] NOT IN ( 'sysdiagrams' ) )

    declare @count int, @ndx int
    declare @script nvarchar(max)
    select @count = count(ID) from @ConstraintsTable
    set @ndx = 1

    while(@ndx <= @count)
    begin
         select @script = DropConstraintScript from @ConstraintsTable where ID = @ndx
         EXEC sp_executesql @script;

         set @ndx = @ndx + 1
    end

    EXEC sp_msforeachtable @command1 = 'TRUNCATE TABLE ?', @whereand = 'AND Object_Id NOT IN (SELECT Object_Id FROM sys.objects WHERE name like ''__Migration%'')';

    set @ndx = 1

    while(@ndx <= @count)
    begin
         select @script = EnableConstraintScript from @ConstraintsTable where ID = @ndx
         EXEC sp_executesql @script;

         set @ndx = @ndx + 1
    end

The script

EXEC sp_msforeachtable @command1 = 'TRUNCATE TABLE ?', @whereand = 'AND Object_Id NOT IN (SELECT Object_Id FROM sys.objects WHERE name like ''__Migration%'')';

Truncates all tables in the database except for the migration table. This is after the drop constraint scripts have been executed. After the tables are truncated, the next part of the script adds all the constraints back in. By using table variables instead of cursors, we should get better performance when the script executes. It may be possible to improve on the script in places to get better performance. This is the only area where we rely on database script execution. By utilizing the benefits of the EF Initializer we ensure the following:

  • The database can be created from scratch if it is not already created.
  • The database is cleansed (all tables truncated) using the script detailed above in the CleanseTables method.
  • The database is then seeded. In my instance I used the same seeding for the test db initializer that I have for my default migration initializer. This ensures that the database is 'reset' to the state that the database was before any data was added in.
Nore Gabbidon
  • 351
  • 1
  • 4
  • 10