A stateless service is a service that does not store any data on the application server. It reads or writes data to the database, returns a value (or not), and after that, any information on the task itself is forgotten.
A stateful service is used to perform transactions, that is a series of tasks that depend on the result of preceding tasks. The easiest example is sending an order at a web store, where you gather your products in a shopping cart, and when you check out, you enter your account data on one page, store it, then enter your billing address, store it, then confirm your order and conclude the transaction. Each step depends on the successful outcome of the preceding step, and data needs to be preserved until the last one of these steps is completed or the transaction is canceled, in which case there has to be a rollback to restore your account balance to the way it was before you checked out.
In most cases, you can implement transactions both ways, but if you were to use stateless services, your client application would have to take care of handling the proper order and completion of tasks, or you would have to find some other way to store the transaction information correctly and manage the rollbacks. That would be a stateful client-side, as you called it.
All of this is, however, quite general, and security and/or session handling would have to be considered in each of those cases. You can very well use session information to authenticate stateless service calls - you will just have to authenticate each call individually, for example by attaching a session id or user id or some other security token to the business data.