Yes, you could do this with a Lambda function, a CloudWatch Metric and a CloudWatch alarm.
Your alarm would be configured to alarm on the metric, if it exceeds some threshold (the threshold being your instance limit).
Your Lambda function, would run on a schedule e.g. every 5 mins, and would do the following:
If the value published to the metric exceeds the threshold of the alarm, it will fire. The lambda function will keep the alarm threshold configured to the limit of instances and will publish datapoints to the metric based on the number of instances currently running.