Hmya, you are getting a picture painted here that's a bit too rosy. Finalizers are not guaranteed to run in .NET either. Typical mishaps are a finalizer that throws an exception or a time-out on the finalizer thread (2 seconds).
That was a problem when Microsoft decided to provide .NET hosting support in SQL Server. The kind of application where restarting the app to solve resource leaks isn't considered a viable workaround. .NET 2.0 acquired critical finalizers, enabled by deriving from the CriticalFinalizerObject class. The finalizer of such a class must adhere to the rulez of constrained execution regions (CERs), essentially a region of code where exceptions are suppressed. The kind of things you can do in a CER are very limited.
Back to your original question, finalizers are necessary to release operating system resources other than memory. The garbage collector manages memory very well but doesn't do anything to release pens, brushes, files, sockets, windows, pipes, etc. When an object uses such a resource, it must make sure to release the resource after it is done with it. Finalizers ensure that happens, even when the program forgot to do so. You almost never write a class with a finalizer yourself, operating resources are wrapped by classes in the framework.
The .NET framework also has a programming pattern to ensure such a resource is released early so the resource doesn't linger around until the finalizer runs. All classes that have finalizers also implement the IDisposable.Dispose() method, allowing your code to release a resource explicitly. This is often forgotten by a .NET programmer but that doesn't typically cause problems because the finalizer ensures it will eventually be done. Many .NET programmers have lost hours of sleep worrying whether or not all Dispose() calls are taken care of and massive numbers of threads have been started about it on forums. Java folks must be a happier lot.
Following up on your comment: exceptions and timeouts in the finalizer thread is something that you don't have to worry about. Firstly, if you find yourself writing a finalizer, take a deep breath and ask yourself if you're on the Right Path. Finalizers are for framework classes, you should be using such a class to use an operating resource, you'll get the finalizer built into that class for free. All the way down to the SafeHandle classes, they have a critical finalizer.
Secondly, finalizer thread failures are gross program failures. Similar to getting an OutOfMemory exception or tripping over the power cord and unplugging the machine. There isn't anything you can do about them, other than fixing the bug in your code or re-route the cable. It was important for Microsoft to design critical finalizers, they can't rely on all programmers that write .NET code for SQL Server to get that code right. If you fumble a finalizer yourself then there is no such liability, it will be you that gets the call from the customer, not Microsoft.