-1
  • Step 1- This is X job that creates the (b) job.dat file
  • Step 2- This is an SSIS package that splits the output dat file into 4 different files to send to Destination
  • Step 3-Moves the four files from the workarea to another location where MOVEIT can pick them up from

***Step two is not restartable
***There is no reversing out if any of the step fails

Note: what if i add exception handler or should I add condional split... any other ideas ?

Sunny
  • 3
  • 3
  • Sooo, if there's no backout plan in case of failure, you should define one. What would it be? Should step 2 be restartable? What would that mean in your specific scenario? What needs to be reversed? Where do you anticipate failures to be induced? – billinkc Jun 19 '15 at 16:09

1 Answers1

0

Batch Persistence

One thing you can do for starters is to append the file(s) names with a timestamp that includes the date time of the last record processed (if timestamps do not apply then you can use a primary key incrementing value). The batch identifiers could also be stored in a database. If your SSIS package can smartly name the files in chronological sequence then third step can safely ignore files that is has already processed. Actually, you could do that at each step. This would give you the ability to start the whole process from scratch, if you must do it that way.

Ignorant and Hassle Free Dumping

Another suggestion would be do dump all data each day. If the files do not get super large then just dump all data for whatever it is you are dumping. This way each step would not have to maintain state and the process could start/stop at anytime.

Ross Bush
  • 14,648
  • 2
  • 32
  • 55