Form submissions lose idempotency when submitted rapidly in a row

Question

I had a problem with people double clicking my form submit button and it submitting multiple times, so I wanted to allow only one submission. My first thought was javascript, but that's not absolutely instantaneous and I wanted something 100% guaranteed to work.

My solution was to make submissions idempotent, by giving each form load its own hash, and on each submission, checking whether that hash already exists, and if it does, not doing anything.

So this is the gist of my code:

#form
@hash = SecureRandom.urlsafe_base64(20)
f.hidden_field :hash, value: @hash
f.submit "submit"

#controller
existing = Submission.find_by(hash: params[:hash])
if existing.nil?
  #enter new Submission into the database with hash: params[:hash]

This works if I allow some time between submissions, but when I do a double click on the submit button, two records are entered into my database, with the same hash value.

I'm using a simple Puma 3.7 server on localhost. I was under the impression that most servers would receive a request, execute it, then move on to the next request, but it's almost like there's some type of parallelism going on.

My question is: how is this possible? Each subsequent record has an ID value one greater than the previous record, so it's not like the server didn't know about the previous record. So how is it that if the requests are sent very rapidly, the unique hash requirement is ignored? Again, if I try again later with the same hash, nothing happens, as expected.

"I'm using a simple Puma 3.7 server on localhost. I was under the impression that most servers would receive a request, execute it, then move on to the next request, but it's almost like there's some type of parallelism going on." That's exactly whats going on. Puma is not single-threaded. — max, Feb 15 '20 at 15:04
@max Obviously, yet the interesting is, why an idempotent check mechanisms fail on permitting insertions, which have the same id/hash-code, seems, that there is a flaw in keeping the database consistent "across" all participating threads ( as if the dirty-cache / dirty-db-data flags propagation fails to keep data consistent for all individual, sequential by nature, atomic db-checks ? ). Btw a trivial problem like a lousy mouse button can simply fire several times the [submit]-event, all upon just one human click, so ***Alea acta est*** — user3666197, Feb 16 '20 at 13:55
@user3666197 I think the main reason this approach is failing is race conditions. Consider if you have two requests that come in a millisecond apart - the first request runs `existing = Submission.find_by(hash: params[:hash])` and but before it has inserted the record the second request also does the same query and finds no records. Then both processes insert a record. This is a well known issue with application validations and can be solved by adding a unique index in the database or by using a transaction. https://thoughtbot.com/blog/the-perils-of-uniqueness-validations — max, Feb 16 '20 at 14:14
@max With all due respect, there ought be zero-doubts about proper transaction-handling, using explicit lock, where needed. Microseconds are not an excuse for ill-formulated test. Transaction-mode update-processing, locking where necessary, protects the consistency of DB-contents, doesn't it? So, the first-comes first-served, 1) lock-DB, 2) test, 3) update, if and only if not present so far, 4) release-lock - this is what makes RDMS data consistent & transaction-safe (w/o illegaly injected DUPes).Here,due to the nature of multi-[submit]-s, a test can lock but a mini-table with most recent data — user3666197, Feb 16 '20 at 18:20
@user3666197 not sure if I completely understood that. Wouldn't locking involve locking the whole table with the the nasty side effect of preventing any other processes from inserting non-dupes? — max, Feb 16 '20 at 18:36
@max Negative, Sir, the small-scale, thus fast *( just a last few tens of seconds of hashes suffice, don't they + right-sizing tuning against sprayed DoS et al artifacts possible in a full context of the target app-domain )* gets locked/tested/unlocked - a circular mini-table is fast and right-enough to be used just for this fast, transaction-safe, non-DUPe confirmations. And once confirmed, the main DB-injection ( if it was permitted ) is now sure to be a non-DUPe, so no side-effects on the main DB-table ( except the here just gained certainty of having but a non-DUPe, idempotent entries ). — user3666197, Feb 16 '20 at 20:18

max · Answer 1 · 2020-02-16T14:35:15.423

Just use Rack::Throttle instead of reinventing the wheel.

# config/application.rb
require 'rack/throttle'

class Application < Rails::Application
  config.middleware.use Rack::Throttle::Interval
end

This applies a 1 second minimum interval between requests from the same client. You can customize this with the min: option.

This can be combined with javascript debouncing/throttling which will further reduce the load on your server (and the client).

I'm using a simple Puma 3.7 server on localhost. I was under the impression that most servers would receive a request, execute it, then move on to the next request, but it's almost like there's some type of parallelism going on.

Puma is multi-threaded. While you can configure it to be single-threaded, Ruby has mostly blocking IO so a single-threaded web server performs poorly under load.

My question is: how is this possible? Each subsequent record has an ID value one greater than the previous record, so it's not like the server didn't know about the previous record. So how is it that if the requests are sent very rapidly, the unique hash requirement is ignored?

Race conditions. What you are doing is an application level uniqueness validation where you run a query against the database and then later insert a row into the database.

If two (or more) processes are responding to duplicate requests at the exact same time both will get a green light by the Submission.find_by(hash: params[:hash]) query and then proceed to insert a record.

This is a well known issue with application validations in any language/framework and can be solved by adding a unique index in the database which will cause the database to reject the duplicate insert statement. This will raise a database driver error in Rails.

But I still believe that this should be handled on the middleware level.

Form submissions lose idempotency when submitted rapidly in a row

1 Answers1