Is there any useful tool that can store history of my folder locally like SVN does? For example I can commit (however not to server but locally) or it can automatically track folder history.
The idea is to view history of my folder and rollback something or so (like in SVN but without server)
This is needes for Windows
Git should do the trick nicely. It is distributed so you can just use it locally without connecting to a server.
What you describes sound a whole lot like the core of distributed/decentralized version control systems (DVCS). The most popular DVCSs are Git and Mercurial (and Bazaar to some degree, though it seems an order of magnitude less widespread).
In fact, I'm doing precisely what you describe for some non-programming projects, with Mercurial (specifically TortoiseHg - the equivalent to TortoiseSVN) and it's working perfectly. Git long had a reputation for inferior Windows support and GUI tools, but I wouldn't be surprised if this has changed. I've never really used Git (wasn't ever dissatisfied with Mercurial, so I only use it for checking out the source of projects that use Git), so I can neither judge its fitness for your usecase nor recomment a frontend.
You can use even SVN (TortoiseSVN) with file:/// protocol for this task (create local repo without server)
