Yes helium can be dangerous, but usually because of other risky behaviour associated with taking it
Statistically speaking it looks as though helium is associated a small, but increasing number of deaths. The following picture is based on mentions of helium on English death certificates (original source this tweet):
But this doesn't help understand whether the problem is that the gas was helium or whether the users were doing something else that created or exacerbated the risk. Since helium is inert and doesn't have any direct physiological affects (though there are some small positive effects, see next reference) it is a little implausible that it is the direct primary cause of death. Helium is used safely in medicine and deep sea diving in gas mixtures that also contain oxygen.
Of course, pure helium will displace oxygen leading to potential asphyxiation. But it isn't obviously easy to achieve that just by breathing it from a balloon. If you inhale enough to become unconscious, you won't breathe any more of the helium and the worst risk is likely to be related to fainting. As soon as you fall you are in air and will regain the required supply of oxygen on refilling your lungs. As this Slate article explains:
You don't have to worry about fatal asphyxiation if you're sucking from a helium balloon at a party. At worst you'll keep going until you get lightheaded and pass out—at which point you'll stop inhaling helium and your body's oxygen levels will return to normal. Of more concern is the possibility that you'll hurt yourself when you fall down.
Industrial accidents involving asphyxiation usually involve closed spaces filled with inert gas. If you faint in one of these, you are still in the dangerous, airless environment and will die quickly if not removed (see this example). It isn't easy to achieve this with helium as it is less dense than almost every other gas so will only fill spaces enclosed that are sealed completely or at the top. It has been done, though. Two students won a Darwin Award for dying after climbing into an 8' helium filled balloon in florida (reported here). In this case, though, monumental stupidity deserves a larger share of blame than helium.
There is another direct risk from trying to inhale helium, but again it isn't the helium that is the problem but the inherent risk of any compressed gas supply. As this bulletin from the Compressed Gas Association explains:
Attempting to inhale helium from a commercial helium balloon filling system poses a greater hazard than does inhaling helium from a balloon. Beyond the risk of passing out, the potential for fatal injury is present. Unfortunately, several young people have been killed while inhaling helium from such a system.
How can a healthy young person be killed by a seemingly harmless substance, you ask? Postmortem examinations of victims explain what occurs, while engineering analysis explains how.
Chemical reaction does not cause fatal injuries. Rather, the pressure of gas inside the lungs is the agent that can kill instantly. Autopsies show that the alveoli (air sacs) in the lungs have been ruptured. Death follows immediately, as the victims literally drown in their own blood. Under such circumstances, cardiopulmonary resuscitation is of no avail.
So, strictly speaking, the risk is gas pressure not helium.
To summarise
There is no physiological effect of helium that makes it risky to inhale.
But, if the user can contrive a situation where the helium completely displaces air, asphyxiation is possible. So don't climb into gas filled balloons or put bags over your head (these are bad ideas whatever the gas.)
Don't breathe any gas direct from the compressed supply, that is very risky because of the effect of uncontrolled pressure on your delicate lungs.
If you must inhale, do it from a small balloon in a room where you won't hurt yourself if you fall and where other, sober, people are present. The risks here are very low.
