1

We have a Spring boot service where we receive files everyday, because of some issue (on producer) we are receiving multiple files with same name & date appended. The new file overwriting the old, to handle it we want to append a sequence (starting from 1) at every file name. But sequence should be auto-reset to '1' at midnight everyday.

Can anybody suggest a API or a way to reset the sequence.

To generate auto sequence, we are using AtomicSequenceGenerator , but we are unable to implement a simple auto-reset logic.

public class AtomicSequenceGenerator implements SequenceGenerator {

    private AtomicLong value = new AtomicLong(1);

    @Override
    public long getNext() {
        return value.getAndIncrement();
    }
}
Shri
  • 57
  • 7
  • 1
    Is there any reason why you can't `value.set(1)` every midnight? – David Soroko Oct 01 '19 at 12:40
  • Similar to what @DavidSoroko suggested just restart the service at midnight – Zack Macomber Oct 01 '19 at 12:41
  • Service restart is not a solution, we may receive a file @ same time . – Shri Oct 01 '19 at 13:02
  • 1
    @Shri And what happens if your service due to unknown causes is down at that time? You all should think about redundancy/availability. It's a bad idea to assume your service will be up and it's possible it will be down due to unknown causes. – Zack Macomber Oct 01 '19 at 13:06
  • @ZackMacomber , we already have kubernetes to handle availability & other issues. We are looking for the better solution to implement Auto-Reset logic. – Shri Oct 01 '19 at 14:25

3 Answers3

3

To not receive twice a 1:

public class AtomicSequenceGenerator implements SequenceGenerator {

    private AtomicLong value = new AtomicLong(1);
    private volatile LocalDate lastDate = LocalDate.now();

    @Override
    public long getNext() {
        LocalDate today = LocalDate.now();
        if (!today.equals(lastDate)) {
            synchronized(this) {
                if (!today.equals(lastDate)) {
                    lastDate = today;
                    value.set(1);
                }
            }
        }
        return value.getAndIncrement();
    }
}

That is a bit ugly, so try a single counter:

public class AtomicSequenceGenerator implements SequenceGenerator {

    private static long countWithDate(long count, LocalDate date) {
        return (((long)date.getDayOfYear()) << (63L-9)) | count;
    }

    private static long countPart(long datedCount) {
        return datedCount & ((1L << (63L-9)) - 1);
    }

    private static boolean dateChanged(long datedCount, LocalDate date) {
         return (int)(datedCount >>> (63L-9)) != date.getDayOfYear();
    }

    private AtomicLong value = new AtomicLong(countWithDate(1, LocalDate.now()));

    @Override
    public long getNext() {
        long datedCount = value.getAndIncrement();
        LocalDate today = LocalDate.now();
        if (dateChanged(dateCount, today)) {
            long next = countWithDate(1L, today);
            if (value.compareAndSet(datedCount+1, next)) {
                datedCount = next;
            } else {
                datedCount = getNext();
            }
        }
        return datedCount;
    }
}

This uses an AtomicLong with the day-of-year packed into the counter.

  • One pulls a next counter.
  • If the date changed then:
  • when one could set the next day's 1, then give it.
  • when not, someone earlier with probably an earlier counter took the 1, and then we need to take the next again.
Joop Eggen
  • 107,315
  • 7
  • 83
  • 138
  • 1
    Funny how you consider your first approach 'a bit ugly'. – bowmore Oct 02 '19 at 07:44
  • @bowmore I can agree to your sentiment. Though shorter it is a combination of two techniques: _atomic data operations_ and _critical region_. Just for style reasons one should go with one. The second solution misses a class AtomicLongWithLevelBreak or such. – Joop Eggen Oct 02 '19 at 08:07
  • I know, I wanted to answer a variation on your first solution, when I noticed yours (and didn't want to basically duplicate your answer), but mine just used a monitor. I ditched the Atomic, using a plain long protected by the monitor and I didn't bother with double checked locking, so didn't need a volatile LocalDate as that was also protected by the monitor. – bowmore Oct 02 '19 at 08:22
  • BTW your second solution has two bugs atm. – bowmore Oct 02 '19 at 08:23
  • @bowmore thanks, just using a monitor is a clean solution and probably shorter; you can still give it as answer. I would like to see it. I did not test my 2nd hacky solution, but two bugs? Maybe you may enlighten us. – Joop Eggen Oct 02 '19 at 09:28
  • 1
    bug1 : `value.compareAndSet(datedCount, next)` is always false because you did `datedCount = value.getAndIncrement()` (the expected value is one higher) – bowmore Oct 02 '19 at 09:32
  • 1
    bug 2 : dateChanged should check for inequality, rather than equality – bowmore Oct 02 '19 at 09:33
  • Posted my own answer as you requested. – bowmore Oct 02 '19 at 09:41
1

You could create a singleton instance of your generator which resets itself as soon as a new date is passed.

Something like this:

public class AtomicSequenceGenerator implements SequenceGenerator {

    // Private constructor in order to avoid the creation of an instance from outside the class
    private AtomicSequenceGenerator(){}

    private AtomicLong value = new AtomicLong(1);

    @Override
    public long getNext() {
        return value.getAndIncrement();
    }

// This is where the fun starts
// The T indicates some type that represents the file date
    private static T prev_file_date = null;
    private static AtomicSequenceGenerator instance = new AtomicSequenceGenerator();

    public static synchronized long getNext(T file_date)
    {
      if ((prev_file_date == null) || (!prev_file_date.equals(file_date)))
      {
        instance.value.set(1);
        prev_file_date = file_date;
      }
      return (instance.getNext());
    }
}
Robert Kock
  • 5,795
  • 1
  • 12
  • 20
1

As requested by @JoopEggen my version of his first solution :

public class AtomicSequenceGenerator implements SequenceGenerator {

    private final Clock clock;
    private final Object lock = new Object();

    @GuardedBy("lock")
    private long value;
    @GuardedBy("lock")
    private LocalDate today;

    public AtomicSequenceGenerator(Clock clock) {
        this.clock = clock;
        synchronized (lock) {
            value = 1;
            today = LocalDate.now(clock);
        }
    }

    @Override
    public long getNext() {
        synchronized (lock) {
            LocalDate date = LocalDate.now(clock);
            if (!date.equals(today)) {
                today = date;
                value = 1;
            }
            return value++;
        }
    }
}

The main differences are :

  • This uses just a private monitor to protect both the LocalDate and the value.
  • value is now a plain long, since it's guarded by a lock, it doesn't need to be AtomicLong anymore.
  • I inject a Clock object (for easier testing)
  • No double checked locking. Arguably double checked locking can be faster, but I don't know if it's really needed, until you do some performance testing.
bowmore
  • 10,842
  • 1
  • 35
  • 43
  • Professional, testable (clock), short. Double checking improves synchronized, though for file uploads a synchronized probably is not the critical point. – Joop Eggen Oct 02 '19 at 09:49