1

I have started recently using SpecFlow and I have 2 basic questions I need to clarify, also to confirm I am on the right way:

  1. As I understand, it is a must that all the input data (test parameters for the scenarios) to be provided by the tester, the same about the test data (input data for the tables involved in the test scenarios)

  2. Are there any existing tools for a quick way of generating test data (inserting it into the DB) ? I am using Entity Framework as part of the Data access layer. I was wondering about some tool that would read the data from a file or probably some Desktop application to provide values for the table's fields (which could also then generate a file from which some other tool could read all the data and generate all the required objects etc).

I also had a look at Preparing data for a SpecFlow scenario - I was thinking if there is already a framework which would achieve insert\delete of test data to use alongside with SpecFlow.

Community
  • 1
  • 1
aly
  • 373
  • 2
  • 7
  • 22

1 Answers1

1

I don't think you are on the right track. SpecFlow is a BDD tool, but in some ways it only covers part of the process. Have a read of http://lizkeogh.com/2013/07/01/behavior-driven-development-shallow-and-deep/ and see if any if the scenarios sound familiar?

To move forwards I would recommend you start with http://dannorth.net/introducing-bdd/ to get a good idea of how it all began. Now lets consider your points;

  1. The tester provides all the test data. Well yes and no. The idea is that between yourself and the feature expert, you are able to have a conversation that provides all the examples that you need to develop your feature. If you don't involve yourself in that conversation, then yes all the data will come from the other side, but the chances are it won't be such high quality as if you are able to ask the right questions and guide the conversation so the data follows a structure that you can code tests too. As an example here, when I first started with BDD I thought I could get the business experts to write the plain text scenario files with less input from the development, but in practice the documents tended to be less useful than when we were involved. Not because they couldn't write decent specifications, but actually because they couldn't refactor them to reuse bindings etc. We were still needed to add our skills into the process.

  2. Why does data go into a database? A good test is isolated to the scope that it is testing. For a UI layer test this means that we don't have a database. For a business tier test we shouldn't be reliant on the database to get data either. In practice a database is one of the most difficult things to include in your testing because once any part of the data changes you cause cascading test failures. Instead I would recommend making your features smaller and provide the data for your test in the scenario or binding. This also makes having your conversation easier, because the fiftieth row of test pack is not something either party is going to remember. ;-) I recommend instead trying to give you data identities, so "bob" might be individual in a test you can discuss, and both sides understand what makes him an interesting example.

good luck :-)

Update: With regard to using a database during testing, my experience is that there are a lot of complexities that make it a difficult choice to work with. Consider these points,

  • How will you reset the state of your data between tests?
  • How will you reset the state if one / some tests fail?
  • If you are using branches or even just if two developers are making changes at the same time, how will you support multiple test datasets?
  • How will you handle two instances of the tests running at the same time (don't forget the build server)?

Have a look at this question SpecFlow Integration Testing with Database Patterns which includes some patterns that you can use.

Community
  • 1
  • 1
AlSki
  • 6,868
  • 1
  • 26
  • 39
  • Hi, I totally agree about the part of not having the test team just write the scenarios and have the developers implement the necessary behaviour (and basically this is how we proceeded). We also do have unit tests for particular layers of the application (so,yes,the business tests use mocked objects). What I don't understand is why it is wrong to have test data specified in one way or another as part of a SpecFlow scenario. One of my acceptance tests is about testing the search on a specific field from a few possible, so there should be some test data provided to make the search against. – aly Jul 23 '13 at 12:25
  • Ok, so assuming you are only using specflow for acceptance testing, then isolation becomes less of an issue. However using a database is still really problematic, it's not "wrong" but you will find it becomes really awkward. I've added some points for you to consider above. – AlSki Jul 23 '13 at 16:51
  • In my point of view, testing also the database is imperative. Otherwise a big portion of the software is left untested. Introduction of such tools as SSDT and LocalDB allow us to store the database schema and data in the version control system and launch an in memory database as required to do isolated test runs. Don't dismiss using a database just because it's hard. – Tuukka Haapaniemi Mar 09 '16 at 13:38
  • Good point, and it sounds like you have a good system there to ease that difficulty. So my only question is, is it worth it? For you it sounds like it is, but when I wrote this (and now), a lot of my database interaction was no more than `select x,y,z from ...` and `insert x`\`update x` as we had moved all our logic out of the Db entirely, removing the stored procedures and what was left didn't require testing. – AlSki Mar 09 '16 at 23:18