How to store files of different types in separate collections using Lithium and GridFS?

Question

I have a model that looks like this:

class Pdf extends \lithium\data\Model
{

    protected $_meta = array('source' => 'fs.files');
    protected $_schema = array('_id'=>array('type'=>'id'));

    public $file;

    /**
    * @param $zipfile string The full name and path of a zip file
    * @param $filename string
    */
    public static function loadFromFile($zipfile, $filename){
        $name = 'zip://'.$zipfile.'#'.$filename;
        $pdf = Pdf::create(); 
        $pdf->file = file_get_contents($name);
        $pdf->filename = $filename;
        $pdf->metadata = ["filename" => $filename, "zipfile" => $zipfile];
        return $pdf;
    }
}

The static function takes the full name of a zip archive and the name of a file in the archive, creates a model, loads content from the archived file, sets some metadata, and returns the model. This is used in a console command I'm writing that will iterate through a list of zip files, each containing a bunch of pdfs, and add the pdfs to the database.

When I save the model, the pdf does indeed get added to the database. I can view all the pdfs from the MongoDB console, i.e. with db.fs.files.find(). I can also get a pdf from the database using the mongofiles command.

However, it would really be nice if instead of storing the pdfs under fs.files I could store them under fs.files.pdfs. This is because I'm planning on also storing jpegs and text documents in the same database.

If I change $_meta in the class to array('source' => 'fs.files.pdfs'), my console command seems to output the pdf document to the terminal instead of adding it (offhand, I'd like to know why Lithium is doing that!)

I've checked the documentation and haven't been able to figure out what I'm doing wrong. Is there any way to get Lithium to store my files in separate collections depending on the model being used?

(My apologies if my terminology is inaccurate; this is my first attempt to use Lithium and MongoDB beyond the basic tutorials.)

FYI, the [GridFS](http://www.mongodb.org/display/DOCS/GridFS+Specification#GridFSSpecification-Specification) terminology for this is a "namespace". — Stennie, Sep 17 '12 at 20:30

score 2 · Accepted Answer · answered Sep 17 '12 at 08:11

However, it would really be nice if instead of storing the pdfs under fs.files I could store them under fs.files.pdfs. This is because I'm planning on also storing jpegs and text documents in the same database.

At the moment, you can't name your source fs.files.pdfs, but pdfs.files and photos.files ...

I've checked the documentation and haven't been able to figure out what I'm doing wrong. Is there any way to get Lithium to store my files in separate collections depending on the model being used?

Take a look to the highlighted lines here: https://github.com/UnionOfRAD/lithium/blob/master/data/source/MongoDb.php#L387-390

Here is how Lithium handles that

score 0 · Answer 2 · answered Sep 28 '12 at 19:23

0

Yeah, I believe we discussed this on IRC. This is a limitation of MongoDB, and I don't believe you can have more than one GridFS prefix (someone correct me if I'm wrong).

If you're just looking for a way to manipulate different file types through different models, I'd recommend creating separate models, then modifying their default queries to include 'type' => 'pdf' in the conditions, or something along those lines.

answered Sep 28 '12 at 19:23

Nate Abele

5,771
32
27

Per my comment on the original question, GridFS does allow multiple prefixes or *namespaces*. "In order to make more than one GridFS namespace possible for a single database, the files and chunks collections are named with a prefix. By default the prefix is fs., so any default GridFS store will consist of collections named fs.files and fs.chunks. The drivers make it possible to change this prefix, so you might, for instance, have another GridFS namespace specifically for photos where the collections would be photos.files and photos.chunks." – Stennie Sep 28 '12 at 20:32
Ah, so it does. Sorry for not reading your original question more carefully. I guess we'll have to patch the adapter. Can you open an issue in GitHub and link to this thread? – Nate Abele Oct 10 '12 at 00:56
Has this since been fixed? I have two separate Models that I would like to store in independent GridFS collections. Adding `protected $_meta = array('source' => 'photos.files');` correctly auto-creates the `photos.files` collection but the chunks end up god-knows-where (possibly nowhere). – Craig McMahon Jun 03 '13 at 09:05

How to store files of different types in separate collections using Lithium and GridFS?

2 Answers2