Should Unit Tests test against the REST API?

Question

For a while now I have unit tested my business logic separately from the REST Api layer.
In the integration tests I've tested the service itself against its api.

Naturally, the integration tests did not include all the edge cases that the unit tests included. It felt like an overkill and duplicate test code.

But in fact, I'm left with a whole layer that is not covered. I can't really be sure that the return value is serialized as expected and that the error codes are correct or if the parameters are deserialized as I want.

My question is, should I just forgo testing the business logic through the objects that implement them, and test through the service in order to incorporate all the cases without duplication of effort?

Notes:

• I don't mean dropping all unit tests. Complex enough units that do not touch the API layer directly should still be tested separately.
• Test runtime is not an issue here.

Update

Added some example pseudo code to clarify

class Book:
  id: int
  title: string


class BookRepository:
  add_book(book: Book) -> Book
  remove_book(id: int) -> bool
  all() -> List[Book]


class BookApi:
  repository = BookRepository()

  @route('/api/books')
  get() -> List[Book]

  @route('/api/books/id', method=POST)
  add(request_body) -> bool :
    book = parse_book_from_request(request_body)
    return repository.add(book)

  @route('/api/books/id', method=DELETE)
  delete() -> bool

Answer 1

From your description, I see the situation as follows:

You have two unit-testing scenarios, namely the unit-testing of the business logic, and the unit-testing of the REST layer. You also have an interaction test scenario, namely the interaction between the REST layer and the business logic. And, you have a subsystem test scenario, namely the test of the subsystem formed by the REST layer and the business logic together.

And, you are concerned about the effort and potential redundancy that might result from having all these test scenarios addressed. (Well, actually you did only mention the business logic unit-tests and integration tests, so I might have made it even a bit worse...)

What can help you here is to ask yourself for each of these testing scenarios, what bugs might still exist that the other tests have not addressed yet. If you think of a test case, but then you can not think of any bug that the test might detect, maybe the test case is not needed or its purpose was already addressed in another way.

Going from bottom to top: You have the unit-tests for the business logic. So, what unit-tests could make sense for the REST layer? You mentioned deserialization, but there may also be plausibility checks and handling of malformed requests. All this needs thorough testing - including negative tests for security reasons. Think about bugs that you would be able to find in the isolated REST layer - these are obviously not found by the unit-tests of the isolated business logic.

Now, entering interaction tests (aka integration tests): The goal of these tests is only on the interaction between the respective parts - not whether any of the parties does the right thing afterwards (which was tested with unit-testing). In other words, the tests check if the right functions are called with the arguments in the right order with the right formats - or, more generally, if both sides of the interface have the same understanding. Boundary cases would make sense here as well - for example to see whether the callee can handle the extreme cases that the caller provides. Admittedly, there is a risk of redundancy here if the interface between the components is large compared to the components size.

You can limit the redundancy, however, in certain ways: Assume your REST layer is designed to filter out invalid book_id values. You want to test if the largest book_id the REST layer would pass on is accepted by the business logic. If the business logic itself checks the book_id, and, if it is not accepted, throws an exception, your interaction test could focus on whether an exception was thrown or not. You would not have to verify that the correct book was found - this was tested when the business logic was unit-tested.

Then, the subsystem tests: Again, think about which bugs might not have been caught that can only be found when looking at the subsystem as a whole. For example, does the subsystem fulfill all requirements or has some feature been forgotten? (Unit-testing will not find forgotten features, interaction testing might - but not in all cases.) Could there be a version mismatch that a test could identify? And, again, try to focus the test on the essential aspects, to avoid redundancy with the unit and interaction tests.

Sorry that I could only give a somewhat abstract advice with abstract examples.