In Windows EC2 instance (tested on Server 2019)
Download and install CloudWatch Agent based on your instance type.
AWS CloudWatch Agent Installation
After that you'll notice a new service called Amazon CloudWatch Agent: it won't start until you configure it.
To configure that run the wizard located at C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-config-wizard.exe
Answer to all questions (you can skip log analysis and choos Basic metric for free storage percentage) and eventually a file called config.json will be created at C:\Program Files\Amazon\AmazonCloudWatchAgent
Move this file to C:\ProgramData\Amazon\AmazonCloudWatchAgent, rename it as amazon-cloudwatch-agent.json and start the service Cloudwatch Agent.
The service should start and collect metrics,check the log to see if there's are any error
If in log located at C:\ProgramData\Amazon\AmazonCloudWatchAgent\Logs you see this...
2020/05/27 16:11:27 I! Config has been translated into TOML
C:\ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.toml
2020-05-27T14:11:31Z I! cloudwatch: get unique roll up list []
2020-05-27T14:11:31Z I! Starting AmazonCloudWatchAgent
2020-05-27T14:11:31Z I! Tags enabled: host=[your host]
2020-05-27T14:11:31Z I! Agent Config: Interval:1m0s, Quiet:false,
Hostname:[your host], Flush Interval:1s 2020-05-27T14:11:31Z I!
cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 23s
2020-05-27T14:11:31Z I! Started the statsd service on :8125
2020-05-27T14:11:31Z I! Statsd listener listening on: [::]:8125
...all's gone well, as long as you have attached a IAM role to your instance to give it permissions to report to cloudwatch.
See this: CloudWatch permissions
Then you can find all your new defined metrics at CWAgent Parameters section in Cloudwatch, so that you can create an alarm