4

I'm currently trying to run some prefect Flows in my local machine (then in some Google cloud runs instances) and I would like to send runs logs to a distant prefect server.

Here's the example code :

@task
def my_task():
    logger = prefect.context.get("logger")

    logger.info("An info message.")
    logger.warning("A warning message.")

@task(log_stdout=True)
def say_hello(name):
    print("Hello, {}!".format(name))

with Flow(name="My First Flow") as flow:

    my_task()
    say_hello(Parameter('msg'))

flow.run(msg='You')

Also, I've put some configs in backend.toml :

backend = "server"

[server]
host = "SERVER_PUBLIC_IP"
port = "4200"
host_port = "4200"
endpoint = "${server.host}:${server.port}"

[server.graphql]
host = "SERVER_PUBLIC_IP"

[server.ui]
host = "SERVER_PUBLIC_IP"
port = "8080"
host_port = "8080"
endpoint = "${server.ui.host}:${server.ui.port}"
graphql_url = "SERVER_PUBLIC_IP:4200/graphql"

[logging]
# The logging level: NOTSET, DEBUG, INFO, WARNING, ERROR, or CRITICAL
level = "INFO"

# The log format
format = "[%(asctime)s] %(levelname)s - %(name)s | %(message)s"

# additional log attributes to extract from context
# e.g., log_attributes = "['context_var']"
log_attributes = "[]"

# the timestamp format
datefmt = "%Y-%m-%d %H:%M:%S"

log_to_cloud = true

So, here's the problem i'm encountering :

When I run this code, I have this error =>

  INTERNAL_SERVER_ERROR: Variable "$input" got invalid value null at
    "input.logs[0].flow_run_id"; Expected non-nullable type UUID! not to be null.

The GraphQL query was:

mutation($input: write_run_logs_input!) {
        write_run_logs(input: $input) {
            success
    }
}

Because the first line of my logs don't have flow_run_id.

Example: {"input": {"logs": [{"flow_run_id": null, "task_run_id": null, "timestamp": "2020-12-22T21:24:13.628595+00:00", "name": "prefect.FlowRunner", "message": "Beginning Flow run for 'My First Flow'", "level": "INFO"....

I'm wondering what did I forget or what am I doing wrong :(

Thanks for help

General Grievance
  • 4,555
  • 31
  • 31
  • 45
AlaeddineB
  • 41
  • 2

1 Answers1

3

There are two types of runs in the Prefect ecosystem:

  • interactive runs: this is the type of run you are attempting here; this is a run that you generate in your local process and are responsible for managing / monitoring. Interactive runs do not know how to talk to a Prefect API, nor do they attempt to. These runs are largely intended for local testing.

  • managed runs: these are runs that the Prefect API knows about, tracks and orchestrates - you can only create these runs via the Prefect API, either through a schedule on a registered Flow or via an ad-hoc created run (which is an API call).

Your code is attempting to mix these two; you are seeing that error because the run you created does not have the appropriate information to communicate with the backend. I suggest checking out some of the Prefect tutorials, maybe starting with this one

chriswhite
  • 1,370
  • 10
  • 21
  • Thanks for answering but you're link give steps about dealing with Prefect Cloud. I think the solution with my issue is to start a local agent and run the flow on that agent. I'm working on it.. – AlaeddineB Jan 11 '21 at 13:47
  • That tutorial covers the steps for either Cloud or Server, including starting an agent. Any steps marked with the "Cloud" identifier can be skipped if you are using Server. – chriswhite Jan 11 '21 at 16:40
  • This answer helped me realize why my registered flows were running but details couldn't be seen on the UI. I had created an interactive run and it's details wouldn't show up on the UI despite fixing things like permissions on storage path, etc. Converting it into a managed run, worked. – Vitalis Oct 17 '21 at 21:46