0

I am working on a project, it has multiple parts in C# and Java(interoperable). I m working with files and it would be nice to keep states of the files. ie: which part did fail and where if there is a failure. also, if everything is processed mark them as processed.

Because I have multiple components for this system, which is about 5 components. I want to be able to use a naming convention to record the files with state names. Lets say File A started component 1, i want to mark it as being at state 1 once it s complete, i want to rename it as A.1.complete and so forth.

I thought about using a persistence database, but it brings more layer and more dependencies to the system. if i can come up with a schema (naming schema), i will avoid the database layer.

How would you do it?

DarthVader
  • 52,984
  • 76
  • 209
  • 300

1 Answers1

2

Create a folder structure for each state

| 1
|- Inbox
|- Working
|- Complete
| 2
|- Inbox
|- Working
|- Complete
| 3
|- Inbox
|- Working
|- Complete
| 4
|- Inbox
|- Working
|- Complete
| 5
|- Inbox
|- Working
|- Complete

Move your files between the different folders when they change state.
Having a folder structure instead of a naming scheme makes it easier to find what items that exists in a given state. Just list all files instead of trying to parse parts of filenames. No risk of going bananas just because your file happened to have the name 'A.1.txt' from the beginning.

Albin Sunnanbo
  • 46,430
  • 8
  • 69
  • 108
  • clever. but what if i have so many files. then i need to create sub folders as well, right? – DarthVader Nov 03 '11 at 05:52
  • How many? Many files in a folder is not a big problem if you just access them from code, it is mostly when you browse the folder in a file manager that things might get slow. – Albin Sunnanbo Nov 03 '11 at 05:58
  • 1
    @DarthVader I did it for you :) – Eng.Fouad Nov 03 '11 at 05:58
  • as much files as to contain about 15 billion delimeted rows. but files will be in chunks. – DarthVader Nov 03 '11 at 05:59
  • Well, that could be a bunch! If you create `x` subfolders you can store files in the folder `GetHash(filename) % x`. Then you will get a pretty even distribution of files an you know where you can find a given file. It is more flexible than beginning letter in how many folders you want. It is also more robust if your files happen to have unexpected starting letters like `-`. – Albin Sunnanbo Nov 03 '11 at 06:07
  • i see what you saying. makes senses. i will think about that. we ll probably use a NAS , big drives. – DarthVader Nov 03 '11 at 06:14