Geting list of protobuf message objects from the SpringBoot

Question

I want to get a list of protobuf message objects from the Spring boot app.
I did manage to get a single protobuf message object from the app but getting a list of them throws exception.

...
2020-01-24 14:57:02.359 ERROR 15883 --- [nio-8081-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet$Parser]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ImmutableCollections$ListN[0]->com.example.demo.Lecture["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])] with root cause

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ImmutableCollections$ListN[0]->com.example.demo.Lecture["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.10.2.jar:2.10.2]
...

My code (simplified).

tl;dr

create Spring boot app
generate class from proto file
try return List of generated class objects (RESTful)

My code (simplified).
Controler

import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.*;
import java.util.List;

@Slf4j
@org.springframework.web.bind.annotation.RestController
@RequestMapping("/timetable")
public class RestController {

    @PostMapping("/single")      // Works
    private Lecture getLecture(@RequestBody Lecture lecture) {
        log.info("Single2 got: {}", lecture);
        return Lecture.newBuilder(lecture)
                .setDuration(lecture.getDuration() +1)
                .build();
    }

    @GetMapping("/list")       // Does not work
    private @ResponseBody List<Lecture> getLectures() {
        return List.of(
                Lecture.newBuilder()
                        .setDuration(1)
                        .setWeekDay(Lecture.WeekDay.MONDAY)
                        .setModule(Module.newBuilder().setName("Math1").build())
                        .build()
                // ... 
        );
    }
}

App

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter;
import org.springframework.http.converter.protobuf.ProtobufJsonFormatHttpMessageConverter;

@SpringBootApplication
public class DemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Bean
    @Primary
    ProtobufHttpMessageConverter protobufHttpMessageConverter() {
        return new ProtobufJsonFormatHttpMessageConverter();
    }
}

pom.xml

<!-- ... -->
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>11</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <!--        https://dzone.com/articles/exposing-microservices-over-rest-protocol-buffers-->
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.11.1</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.protobuf-java-format</groupId>
            <artifactId>protobuf-java-format</artifactId>
            <version>1.4</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java-util -->
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java-util</artifactId>
            <version>3.11.1</version>
        </dependency>
    </dependencies>
<!-- ... -->

I generate message objects using:

#!/bin/bash
SRC_DIR=../proto
DST_DIR=../../../target/
mkdir -p $DST_DIR
protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/college.proto

proto file

syntax = "proto3";

package my.college;

option java_multiple_files = true;
option java_package = "com.example.demo";

message Module {
    string name = 1;
    // ... other
}

message Lecture {
    WeekDay weekDay = 1;
    Module module = 2;
    uint32 duration = 3;
    // ... other

    enum WeekDay {
        SUNDAY = 0;
        MONDAY = 1;
        // ...
    }
}

I did found simmilar issue but it had no solution.

score 2 · Answer 1 · answered Nov 09 '21 at 19:09

Explanation

Spring will choose the HttpMessageConverter that matches the appropriate converter for the object type of your response body. In this case, it is likely choosing MappingJackson2HttpMessageConverter of the ProtobufJsonFormatHttpMessageConverter because your response body has type List.

MappingJackson2HttpMessageConverter implements HttpMessageConverter<Object> ProtobufJsonFormatHttpMessageConverter implements HttpMessageConverter<Message>

Since ProtobufJsonFormatHttpMessageConverter does not support serializing a List type, we can instead tell the MappingJackson2HttpMessageConverter how to serialize the Message type through configuration.

Solution

A bean of type Jackson2ObjectMapperBuilderCustomizer can be used to register a serializer for Message types.

@Bean
public Jackson2ObjectMapperBuilderCustomizer jackson2ObjectMapperBuilderCustomizer() {
    return o -> o.serializerByType(Message.class, new JsonSerializer<Message>() {
        @Override
        public void serialize(Message value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
            gen.writeRawValue(JsonFormat.printer().print(value));
        }
    });
}

score 0 · Accepted Answer · answered Jan 28 '20 at 21:53

Workaround
I couldn't find a solution to the problem so came up with a workaround.
Instead of returning generated protobuf message objects I returned wrappers for those objects. Using Lombok annotation it could be done:

import lombok.Data;

@Data // Lombok magic
public class Module {
    private String name;
    // ...

    public Module(ie.gmit.proto.Module moduleProto){
        this.name = moduleProto.getName();
        // ...
    }
}

This workaround doesn't feel very bad as it uses standard Spring boot dependencies.

Geting list of protobuf message objects from the SpringBoot

tl;dr

2 Answers2