'Stateless' means that the server-side application isn't keeping information about individual clients across calls to it. Many applications keep information in the HTTP session (maintaining conversational state or caching things that are likely to be needed again), a stateless application would not do this. The client side can make calls to the server side and keep state locally.
Stateless is good because it means any server can service any request, without having to resort to clustering (where HTTP sessions have to be replicated across servers this bogs down as the number of servers increases) or sticky sessions (sending requests to the same server where the user started a session, so a server's load can get unbalanced easily). With no state the requests can be distributed across the servers more evenly, and if one goes down it's less of a problem.
The server can expose data through web service calls using REST or SOAP. There are a lot of frameworks to help you expose services whether you use Spring or pure Java EE. The client can call these services and maintain a local model within the browser as part of the DOM using AngularJs. Having services return data as JSON makes them more convenient for the client-side JavaScript to consume.