I was waiting for somebody else to step in, as I am not proficient in mysql, and I believe there must be a better solution.
Main problem here is a way to construct a table of hours in a day. Having no recursive select ability in mysql forced me to create a table by means of union. Dates are easier if we can accept missing days when nobody logged in at that date. If not, similar trick as with hours can be used to extend dates, for example for seven days.
Cross join will produce a table of dates, each having all 24 hours of the day. Now we need to count sessions that were active at this point of time. To do that we need to truncate startTime to hour boundaries and place cross-join combined time inside truncated startTime and endTime (which does not need a truncation). Our data is finally here.
To get average for last two months simply wrap this select in another one grouping hour and calculating avg(Users). If you really must have single query to return both datasets, you might union this query with average query, where average query would return null for date.
Additional disclaimer: as a stated earlier I do not know MySql. I tried to write date and time conversion functions using online manual. Probably failed miserably, but I believe you will correct me. I'm also not sure about reserved keywords.
select days.date,
hour,
count (s.startTime) Users
from
(
(
select 0 hour
union
select 1 hour
union
select 2 hour
union
select 3 hour
union
select 4 hour
union
select 5 hour
union
select 6 hour
union
select 7 hour
union
select 8 hour
union
select 9 hour
union
select 10 hour
union
select 11 hour
union
select 12 hour
union
select 13 hour
union
select 14 hour
union
select 15 hour
union
select 16 hour
union
select 17 hour
union
select 18 hour
union
select 19 hour
union
select 20 hour
union
select 21 hour
union
select 22 hour
union
select 23 hour
) hours
cross join
(
-- We need date portion only
select distinct date(startTime) date from s
union select distinct date(endTime) from s
) days
)
left join s
-- date+hour, hopefully
on date_add(date, interval hour HOUR)
-- startTime is truncated to hour, hopefully
between date_sub(s.startTime interval minutes(s.startTime) MINUTE)
and s.endTime
-- last two months
where days.date between date_sub (now() interval 2 MONTH) and now()
group by days.date, hour
order by 1, 2