0

I am trying to generate python source code from flatbuffer schema and using the generated code in another application. But it just doesn't seem to work.

I have created a simple reproducible code that highlights the issue that I am facing. Following is the directory structure:

-dummy/
      - BUILD
      - main.py
      - sample.fbs
      - WORKSPACE

What I am trying to do here is this:

  • Define a structure in sample.fbs
  • Generate python source code corresponding to the defined flatbuffer schema
  • Use the resulting generated source code in main.py

The contents of each of the above files are as follows:

  • BUILD
    load("@com_github_google_flatbuffers//:build_defs.bzl", "flatbuffer_library_public")
    
    package(default_visibility = ["//visibility:public"])
    
    python_export_classes_list = [
        "__init__",
        "Foo",
    ]
    
    flatbuffer_library_public(
        name = "schema_py",
        srcs = ["sample.fbs"],
        outs = ["py/%s.py" % f for f in python_export_classes_list],
        language_flag = "-p",
        out_prefix = "py/",
    )
    
    py_library(
        name = "schema_lib",
        srcs = [":schema_py"],
        imports = ["py/"],
    )
    
    py_binary(
        name = "main",
        srcs = ["main.py"],
        deps = ["//:schema_lib"]
    )
  • main.py
    from foo import Foo
  • sample.fbs
    namespace foo;
    
    struct Foo {
        bar: int;
    }
  • WORKSPACE
    workspace(name = "dummy")
    
    load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
    load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")
    
    maybe(
        git_repository,
        name = "com_github_google_flatbuffers",
        remote = "https://github.com/google/flatbuffers",
        commit = "a9a295fecf3fbd5a4f571f53b01f63202a3e2113", # flatbuffers 2.0.0
    )

The problem I am facing is this:

Whenever I try to run the target //:main using the command bazel run //:main, I get the following error:

DEBUG: Rule 'com_github_google_flatbuffers' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1620672316 -0700"
DEBUG: Repository com_github_google_flatbuffers instantiated at:
  /home/abc/dummy/WORKSPACE:7:6: in <toplevel>
  /home/abc/.cache/bazel/_bazel_abc/8ad323a425180f6fe9a223b33e8e7665/external/bazel_tools/tools/build_defs/repo/utils.bzl:201:18: in maybe
Repository rule git_repository defined at:
  /home/abc/.cache/bazel/_bazel_abc/8ad323a425180f6fe9a223b33e8e7665/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
INFO: Analyzed target //:main (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/abc/dummy/BUILD:10:26: declared output 'py/__init__.py' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /home/abc/dummy/BUILD:10:26: declared output 'py/Foo.py' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely)
ERROR: /home/abc/dummy/BUILD:10:26: Generating flatbuffer files for schema_py: //:schema_py failed: not all outputs were created or valid
Target //:main failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/abc/dummy/BUILD:24:10 Middleman _middlemen/main-runfiles failed: not all outputs were created or valid
INFO: Elapsed time: 0.097s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

I am just not understanding why the python source code files from the schema are not getting generated.

Any help would be greatly appreciated.

My bazel version:

Build label: 4.2.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Aug 30 15:17:47 2021 (1630336667)
Build timestamp: 1630336667
Build timestamp as int: 1630336667

OS: Linux Ubuntu 20.04.3 LTS

harman
  • 333
  • 2
  • 11

1 Answers1

0

If I'm reading the flatbuffer python generator correctly, I think it's generating py/foo/__init__.py, but not py/__init__.py. Try changing outs to include that. I think you're expected to create/generate py/__init__.py yourself.

Brian Silverman
  • 3,085
  • 11
  • 13
  • There is no luck with changing `outs` to `outs = ["py/foo/%s.py" % f for f in python_export_classes_list]` since it throws that `py/foo/Foo.py` was not created by genrule. – harman Jan 04 '22 at 04:40