Jpa repository and JsonIgnoreProperties return the same object differently in getOne() and in findAll() methods

Question

Brief background

I have two methods in an API created on Spring Boot to retrieve data from a mySQL database (via Hibernate and JpaRepository). There is a method that returns all the ocurrences in a table called test, and another one that returns the test corresponding to the id passed as a parameter in the GET call. Both API entry points (by means of the services and repositories) end up calling two JpaRepository methods (findAll() and getOne() respectively).

Problem description

The problem I'm observing is that in the findAll() method, JpaRepository behaves differently from getOne() when it comes to returning lists of internal objects corresponding to a @ManyToMany relationship in the model. The test class has a list of requisite objects corresponding to a different entity of the model. Both classes are as follows:

Pectest.java

@Entity
@JsonIgnoreProperties({"hibernateLazyInitializer", "handler"})
@JsonIdentityInfo(generator=ObjectIdGenerators.PropertyGenerator.class, property="idtest")
public class Pectest {
@Id
@Column(name="idtest")
private long idtest;
private String testname;
private String testdescription;

[...]
// Other properties and fields
[...]

@ManyToMany(fetch = FetchType.LAZY
    )
@JoinTable(name = "test_checks_requisite",
        joinColumns = @JoinColumn(name = "test_idtest"),
        inverseJoinColumns = @JoinColumn(name = "requisite_idrequisite")
    )
@JsonProperty(access=Access.READ_ONLY)
private Set<Requisite> requisites = new HashSet<Requisite>();
[...]

Requisite.java

@Entity
@JsonIgnoreProperties({"hibernateLazyInitializer", "handler"})
@JsonIdentityInfo(generator=ObjectIdGenerators.PropertyGenerator.class, property="idrequisite")
public class Requisite {
@Id
@Column(name="idrequisite")
private long idrequisite;
private String Name;
private String Title;
private String Description;
@ManyToOne(cascade = {CascadeType.PERSIST}, fetch=FetchType.LAZY)
@JoinColumn(name="functionality_idfunctionality")
@JsonProperty(access=Access.WRITE_ONLY)
private Functionality functionality; 

@ManyToMany(mappedBy="requisites")
private Set<Pectest> tests = new HashSet<Pectest>();

Here goes a fragment of the JSON returned by the GET mapping that returns all tests. It can be observed that the first test object, the one with idtest 9 has a collection of 5 requisite objects under the requisites property. The second test (with id 10) on the contrary, only shows the full requisite object in the one which doesn't exist in the previous one, showing only the id's value in the others:

[
{
    "idtest": 9,
    "testtype": {
        "idTestType": 5,
        "testTypeName": "Manual"
    },
    "datatype": null,
    "requisites": [
        {
            "idrequisite": 7,
            "name": "REQ-0006",
            "description": "Campo para traducción de nombres, debe mostrar el nombre en el idioma seleccionado por el cliente.",
            "title": "DisplayName"
        },
        {
            "idrequisite": 4,
            "name": "REQ-0003",
            "description": "Se ofrece acceso a parámetros a través de este tipo de nodo",
            "title": "Parameter Type"
        },
        {
            "idrequisite": 5,
            "name": "REQ-0004",
            "description": "El BrowseName de las variables debe ser sólo el último campo de las variables exportadas por FW de BR (ItemName)",
            "title": "BrowseName"
        },
        {
            "idrequisite": 3,
            "name": "REQ-0002",
            "description": "Se ofrece acceso a variables analógicas a través de este tipo de nodo",
            "title": "Analog Type"
        },
        {
            "idrequisite": 2,
            "name": "REQ-0001",
            "description": "Se ofrece acceso a variables digitales a través de este tipo de nodo",
            "title": "Digital Type"
        }
    ],
    "testDescription": "El servidor es capaz de devolver correctamente todos los tipos de dato.",
    "lastUpdated": null,
    "testName": "Lectura de DataTypes",
    "dateCreated": null,
    "testURL": ""
},
{
    "idtest": 10,
    "testtype": {
        "idTestType": 5,
        "testTypeName": "Manual"
    },
    "datatype": {
        "idDataType": 2,
        "dataTypeName": "Boolean"
    },
    "requisites": [
        7,
        5,
        {
            "idrequisite": 10,
            "name": "REQ-0009",
            "description": "Se generan a partir de los niveles de acceso de la variable. Se deben escalar los valores, de 256 a 1000.",
            "title": "AccessLevel & UserAccessLevel"
        },
        2
    ],
    "testDescription": "El servidor es capaz de admitir escrituras correctamente de todos los tipos de dato.",
    "lastUpdated": null,
    "testName": "Escritura de DataTypes",
    "dateCreated": null,
    "testURL": ""
}
...
]

And here goes the result of calling the GET method for a single object via its id (in this case the one with id 10, which is incorrect in the previous JSON):

{
"idtest": 10,
"testtype": {
    "idTestType": 5,
    "testTypeName": "Manual"
},
"datatype": {
    "idDataType": 2,
    "dataTypeName": "Boolean"
},
"requisites": [
    {
        "idrequisite": 7,
        "name": "REQ-0006",
        "description": "Campo para traducción de nombres, debe mostrar el nombre en el idioma seleccionado por el cliente.",
        "title": "DisplayName"
    },
    {
        "idrequisite": 2,
        "name": "REQ-0001",
        "description": "Se ofrece acceso a variables digitales a través de este tipo de nodo",
        "title": "Digital Type"
    },
    {
        "idrequisite": 5,
        "name": "REQ-0004",
        "description": "El BrowseName de las variables debe ser sólo el último campo de las variables exportadas por FW de BR (ItemName)",
        "title": "BrowseName"
    },
    {
        "idrequisite": 10,
        "name": "REQ-0009",
        "description": "Se generan a partir de los niveles de acceso de la variable. Se deben escalar los valores, de 256 a 1000.",
        "title": "AccessLevel & UserAccessLevel"
    }
],
"testDescription": "El servidor es capaz de admitir escrituras correctamente de todos los tipos de dato.",
"lastUpdated": null,
"testName": "Escritura de DataTypes",
"dateCreated": null,
"testURL": ""

}

Additional information

I have observed the following behavior:

If I associate any requisite just to a single test, it works as intended.
If I associate a requisite to two or more tests, it will only return the object on the first one, returning only its index value in the following ones.

Because of those two points, I believe that it may seem like some caching issue (I haven't enabled second level cache nor query level cache). I mean, it seems like hibernate only retrieves a given requisite from the database the first time, and for the rest of them detects that a requisite with that id has been requested and doesn't launch the query. Which is good, of course, but how can I have the cached objects in my results in the subsequent test objects that have that requisite associated?

Edit: Ok, I post this as an answer, because the problem I describe in my original post seems to disappear. As I was saying in the comments to the original post, I removed the @JsonIdentityInfo annotation on the Requisite class. I was using it to avoid infinite recursion due to @ManyToMany relationships between model classes.

However, I don't think this is a solution, and definitely I don't understand how @JsonIdentityInfo works. I'll read the documentation, but I'd appreciate an explanation if somebody can provide one.

(1) There is no `JpaRepository.listAll` method, I think you meant `findAll` (2) `getOne` is **not supposed to be used for retrieving the state of an entity**. Use `findById` instead — crizzis, Oct 29 '18 at 15:16
You're right, listAll() is the name of my method in the service interface I created. The JpaRepository method is findAll(), I'll edit the question to clarify this, thanks. Regarding to the getOne, I'll switch to findOne (due to what I saw in the accepted answer in [this thread](https://stackoverflow.com/questions/24482117/when-use-getone-and-findone-methods-spring-data-jpa) about being more generic) but that is the method that actually works fine in my case. I'll post the results anyway. Thank you for your answer! — Eme, Oct 29 '18 at 15:31
It is not about one being more 'generic' than the other. The only valid use for `getOne` is when you want to *establish a relationship between entities* (e.g. `pet.setOwner(ownerRepository.getOne(ownerId))`. In such a use case, neither you nor JPA needs to access any state of `Owner` other than the id. For all the other purposes, you should use `findOne/findById` — crizzis, Oct 29 '18 at 15:36
Your question is more about the `@JsonIgnoreProperties` than anything about spring-data-jpa. You are showing what is happening with the json output in regards to children entities that you haven't specifically fetched because they are market with FetchType.LAZY. I'd initially say the results are undefined because of this though I don't know what exactly to expect from the Json Serialization library. Granted the outputs are different but if you're interested in the children you should be sure to include that in your fetch query. — K.Nicholas, Oct 29 '18 at 17:18
You're using the annotation JsonIdentityInfo. The whole point of this annotation is to tell Jackson to just use the ID of an object instead of serializing the object if it has already been serialized before. If you don't want that, why did you choose to use that annotation? — JB Nizet, Oct 29 '18 at 17:25
@crizzis Thank you for the explanation, I see the difference now and I've changed it. I'll refactor to take advantage of the Optionals. — Eme, Oct 30 '18 at 07:40
@K.Nicholas Thanks for the answer. I had tried before with FetchType.EAGER, and it makes no difference in the behavior I describe, so my guess is that it's unrelated to this particular issue. And I usually prefer lazy fetch to avoid unnecesary load, so that's why I left it that way. I tried again, though, and I can confirm that nothing changes. — Eme, Oct 30 '18 at 07:58
The reason `FetchType.LAZY` makes no difference is likely because the serialization library tries to serialize `Pectest.requisites`, thus triggering the lazy loading (which I think is what @K.Nicholas was trying to tell you) — crizzis, Oct 30 '18 at 08:02
@JBNizet I see your point, and maybe that's why all of this is happening. I used JsonIdentityInfo to avoid infinite recursion problems in the ManyToMany associations (**Pectest** class has a list of **Requisite**, and **Requisite** has a list of **Pectest**), so this makes a lot sense as the root of my problem. Maybe I should use JsonIdentityInfo just on **Pectest** class? — Eme, Oct 30 '18 at 08:04
IMHO, you should stop using entities, having circular dependencies and designed for the persistence layer of your application, as objects you serialize to JSON as responses of the web service layer of your application. Decide what exactly each web service should return as JSON, create a class matching that JSON structure, and transform the entities into instances of these classes. — JB Nizet, Oct 30 '18 at 10:21
This is probably offtopic but I'm using my Entity classes as a reflection of the database to be accessed and manipulated from the REST API. Maybe not in every project can be like this, but in my case the database tables already match what I want to serialize as a JSON to be accessed from the outside world. So I didn't see the point of creating an intermediate class (and the associated code to convert between them). Is using entities this way a bad practice? — Eme, Oct 30 '18 at 11:44
It's not exactly a bad practice (no need to overcomplicate things), but you should be aware that there are caveats: (1) if you want to return additional fields in one request and exclude them in another, this approach breaks: (2) sometimes, JSON serialization libraries may trigger side effects (e.g. issuing additional queries). At a minimum, I'd recommend disabling `spring.jpa.open-in-view` and making sure the transaction only spans the service method (excluding the controller) so as to make sure that when the entities are getting serialized, they are already detached from the context — crizzis, Oct 31 '18 at 10:37
Thanks for the explanation, I'm using Jackson and I'm definitely having effect (2), so I'll keep all of this in mind. — Eme, Nov 05 '18 at 08:40