I am struggling a little bit to understand how to implement a Hystrix Metrics Publisher plugin.
Having read the documentation, it is still not clear how things are supposed to work together.
My goal is to write a plugin that will collect every metrics published by hystrix and write these metrics to a file on disk. This file will later be collected and processed by an external tool giving us a good historical basis of the circuit’s behavior and problems.
The system where hystrix is running is a normal spring application. This said, I am also somewhat new on the java platform (although I am comfortable with the java language).
I thought that a first step towards understanding how the plugin could be implemented would be looking at the already implemented publishers. With this in mind, I looked at some of the implementations of hystrix-contrib directory.
I have chosen hystrix-codahale-metrics-publisher and hystrix-servo-metrics-publisher.
Both of them have a main class (servo is HystrixServoMetricsPublisher) which seems to register for receiving all possible kinds of metrics and some classes to deal with each kind of metric.
By looking at what I will call the main class, I see that, for example, there is a method called getMetricsPublisherForCommand that must return an implementation of the interface HystrixMetricsPublisherCommand.
Now questions start:
Question 1 I am assuming that once a plugin is registered every execution of every command on the context where the plugin is registered, and by the word command we can understand every execution of the execute() method of every class which inherits from HistrixCommand on that context, will generate a call to the getMetricsPublisherForCommand() method of my plugin. Is it true? If so, there are a lot of low level implementations in hystrix such as thread pools and other, Should my getMetricsPublisherForCommand() implementation be thread-safe or I am guaranteed to receive calls in a sequential order? On what thread will my getMetricsPublisherForCommand() be executed?
Question 2 By looking at the documentation I am still not sure about what exactly the implementation of HystrixMetricsPublisherCommand to be returned by getMetricsPublisherForCommand() has to do. This is due to the fact that the HystrixMetricsPublisherCommand interface only specifies a method called initialize (). if it specified a method called, say, publish() I would conclude that the hystrix engine would call my custom getMetricsPublisherForCommand() method to get a metrics publisher on which it would call a publish() method to perform the custom publishing. But the initialize () method seens to be called only once when this given object is returned and I have found no other method the engine would call afterwards.
Also, by reading the documentation, I am under the impression that the implementation of HystrixMetricsPublisherCommand returned by getMetricsPublisherForCommand() will be somehow a singleton which completely breaks my understanding about how the thing is supposed to work.
The documentation say this:
The initialize() method will be called once-and-only-once to indicate when this instance can register with external services, start publishing metrics etc.
If you look at the servopublisher however you will notice that, unless I am completely and absolutely confused, the publishing stuff is performed right from the constructor. Now, if initialize() will be called to make some setup, how can I code my logic from the constructor where, unless the object is a singleton, it will be executed before any method including initialize () will have a chance to be called? In the other hand ,,, if this is a singleton, how can it run its constructor for every hystrix command?
May be I have missed something, I don't know ... but I need to understand conceptually what is going on here in order to implement my logic the right way. Thanks for your patience and I hope I have made myself clear enough in this long question.
