0

I started using fog storage for a project. I do the most simple actions: upload an object, get the object, delete the object. My code looks something like this:

storage = get_storage(...) // S3 / OpenStack / ...
dir = storage.directories.get(bucket) # 1st request
if !dir.nil?
  dir.files.create(key: key, body: body) # 2nd request

  # or:
  dir.files.get(key) # 2nd request

  #or
  file = dir.files.get(key) # 2nd request
  if !file.nil?
    file.destroy # 3rd request
  end
end

In all cases there's a 1st step to get the directory, which does a request to the storage engine (it returns nil if the directory doesn't exists).

Then there's another step to do whatever I'd like to do (in case of delete there's even a 3rd step in the middle).

However if I look at let's say the Amazon S3 API, it's clear that deleting an object doesn't need 3 requests to amazon.

Is there a way to use fog but make it do less requests to the storage provider?

Gavriel
  • 18,880
  • 12
  • 68
  • 105

1 Answers1

1

I think this was already answered on the mailing list, but if you use #new on directories/files it will give you just a local reference (vs #get which does a lookup). That should get you what you want, though it may raise errors if the file or directory does not exist.

Something like this:

storage = get_storage(...) // S3 / OpenStack / ...
dir = storage.directories.new(key: bucket)

dir.files.create(key: key, body: body) # 1st request

# or:
dir.files.get(key) # 1st request

#or
file = dir.files.new(key)

if !file.nil?
  file.destroy # 1st request
end

Working in this way should allow any of the 3 modalities to work in a single request, but may result in errors if the bucket does not exist (as trying to add a file to non-existent bucket is an error). So it is more efficient, but would need different error handling. Conversely, you can make the extra requests if you need to be sure.

Nowaker
  • 12,154
  • 4
  • 56
  • 62
geemus
  • 2,522
  • 16
  • 22
  • 1
    Updated with code, though I haven't tested this code so there might be minor errors, it should hopefully make it clearer what the suggested solution was. – geemus Sep 25 '20 at 13:25
  • Interesting, replacing `.get(bucket)` with `.new(bucket)` results in `Initialization parameters must be an attributes hash, got String` – Nowaker Oct 01 '20 at 23:26
  • 1
    Ah, it should probably be `.new(key: key)` or something similar. – geemus Oct 02 '20 at 21:58
  • Okay, I'll try `storage.directories.new(key: bucket)` and we'll see. I remember trying `.new(bucket_name: bucket)` and that didn't work but I wasn't sure about the hash parameter name. – Nowaker Oct 03 '20 at 22:31
  • 1
    Awesome, glad that helped. Sorry I missed that difference the first time around. – geemus Oct 06 '20 at 21:02