6

I am creating a command line utility using go , which will request particular server for data and will get the data from server in JSON response. Utility can do multiple requests for multiple products . I am able to do multiple requests and get the response for each product properly in JSON format. Now , I want to save the result locally in caching or local files. By which on every request I will check the local cache before sending request to server . If data is available for that product then no request will be send .

I want to save the whole json response in cache or local file and keep that data every time before doing any request to server for data.

Use Case : Products {"A","B","C","D","E"} It could be any number of products Test 1 : Get data for A,B,C Check local storage whether data is available or do request. Save json request in storage for each product.

So for test 1 ,It will have entry like: [{test 1 : [{product a : json response} , {product b : json response} ,{product c : json response}]}]

And in case if test fails in between and it get results for two products it should save response for 2 products and if we reran the test it will get result for 3rd product only.

rohan
  • 161
  • 1
  • 1
  • 10
  • If this program has a clearly defined (and synchronous) end state, i.e. that it knows request 1 & 2 succeeded and request 3 failed, is a local .json file out of the question (perhaps noting that request 3 failed in the instance given above)? Then it could feasibly read the .json file back into memory on a different run and understand the state from the prior execution. This also allows your deployment to match what you stated, be a single go binary that can be be distributed stand alone. It will produce it's own local data-store, and be somewhat dependent on it, but it isn't a second program. – K. Rhoda Sep 19 '18 at 21:28

2 Answers2

1

There's a bunch of Go libraries to do HTTP caching transparently.

Choose the one you like most and satisfies your needs better, both have examples in their README to get you started real fast.

If for some reason you can't or don't wanna use third-party libraries check out this answer https://stackoverflow.com/a/32885209/322221 which uses httputil.DumpResponse and http.ReadResponse, both on Go's standard library and also the answerer provides an example implementation you can base your work on.

Community
  • 1
  • 1
toqueteos
  • 771
  • 1
  • 6
  • 14
  • Is it possible to store the results in key value pair in file and fetch it and parse it when required. – rohan Jun 24 '16 at 12:22
  • Yes, it's possible. Example: https://github.com/gregjones/httpcache provides multiple backends right out of the box, there's diskcache and memcache for example that provide just that. – toqueteos Jun 27 '16 at 07:32
  • But this goes a step forward and does caching automatically for you. You just tell it where to save things and then perform regular http calls. – toqueteos Jun 27 '16 at 07:33
0

You can store the data inside a Map and get it via Key, you can implement it or use plugin such as go-cache.
As alternative you can use Redis for storing the data, here you can find the driver for Go

Tinwor
  • 7,765
  • 6
  • 35
  • 56
  • How can we save cache in file using go-cache , there is no example for saving results and fetching cache from file – rohan Jun 24 '16 at 12:23
  • @rohan using go-cache you can only store in memory your value. If you want store in a file the best choice is using Redis – Tinwor Jun 24 '16 at 12:28
  • Redis will require its own server or installation in Go also , correct ? As it is required in other languages , Sorry I am new to Golang. As the tool will be command line and it will be distributed to users as single go file which they can build on their systems. I am trying to avoid any extra installation required for tool, How good it will be to store Map objects in files locally and read and write in file only. Thanks in Advance – rohan Jun 25 '16 at 18:12