Is there a quicker way to get workflowHistory

Question

we use cadence 0.11

In our project, there is an often used service which needs the workflowHistory.
So we have to call this function oftenly:

GetWorkflowHistory(ctx context.Context, workflowID string, runID string, isLongPoll bool, filterType s.HistoryEventFilterType) HistoryEventIterator

The problem is, the function calling is quite slow and it makes our service quite slow. What's more, we must ensure the correctness of the workflow so we can not use cache since cache cannot store data which updates frequently.

Is there a quicker way for us to get workflowHistory? maybe a new api, or some new configurations in cadence?

What is your use case of calling the API programmatically? It's quite unusual to use that. Typically, the API needs to load history from DB and the latency depends on the length of the history. — Long Quanzheng, Sep 01 '21 at 03:39
thanks. we use it to get whole `historyEvents` `historyEvents` will be parsed and its information will be used to contribute a **flowChart** . In our logic, flow chart must be Real-time display. What puzzles me is that although the length of history is short(less than 30), it also may took 50ms-5s to get it @Long Quanzheng — vox_vox, Sep 01 '21 at 05:40

Long Quanzheng · Answer 1 · 2021-09-02T21:15:46.530

There is no faster way to get the history. The latency depends on the length and size of the history. This is a critical api in Cadence so the perf has been optimized to the best within its architecture.

However, based on your use case, this is a misuse of the api. You should implement Query handler for the workflow. Worker will do everything same as you are doing to parse history.

If preferred, you can implement a very generic Query handler which return almost everything in the workflow events. So that only one Query handler can let you get anything like getting from the history. Specifically, you can put all signals, activity input output etc into a list and return the list as the query results.

Using worker query will save the latency for you. This is because Cadence worker would cache the history. When there is no changes on the workflow it will not get any more history from server. when new events are appended, only delta of events will be transported between the worker and server. Therefore the latency is minimum regardless of the length of the history.

The only case that worker will need to reload the whole history is when cache is evicted or worker restart. so it’s always recommended to keep history short so that it won’t cost too much resources in that case.

See more docs about query.

https://cadenceworkflow.io/docs/concepts/queries/#stack-trace-query

Golang

https://cadenceworkflow.io/docs/go-client/queries/

Java https://cadenceworkflow.io/docs/java-client/queries/

Taking Golang as an example, if you want to return activity result as the query output:


func MyWorkflow(ctx workflow.Context, input string) error {
    var res map[string]string 
    err := workflow.SetQueryHandler(ctx, "current_state", func() (map[string]string, error) {
        return res, nil
    })
    
    ctx = workflow.WithActivityOptions(ctx, ...)
    var act_out string 
    err = workflow.ExecuteActivity(ctx, ActivityA, "my_input").Get(ctx, &act_out)
    if err != nil {
        res["ActivityA"] = act_out
        return err
    }
    return nil
}

Thanks for your answer we used to get all the information straightforwardly from cadence server, now it seems we need to record all the changes or information in database or cache by ourselves if we use Query, orz thanks again！ — vox_vox, Sep 01 '21 at 08:06
No, you don’t need to use any database or cache if using Query. Since you were able to parse the history to get the information for your use case, technically you can get all information directly from Query too. Can you tell me an example of what information you are parsing? I can write a sample code of Query to show you — Long Quanzheng, Sep 01 '21 at 14:07
we often get the `historyEvents` from cadence-server and parse it to get the `ActivityTaskCompleted` events so that we can find which tasks have been finished. How can I get this information using Query ? thanks. — vox_vox, Sep 02 '21 at 05:52

Is there a quicker way to get workflowHistory

1 Answers1