Stack Overflow
Stack Overflow

Questions tagged [gridfs]

GridFS is a specification for storing large files in MongoDB. All of the mongodb.com supported drivers implement the GridFS spec.

GridFS is a specification for storing large files in MongoDB. MongoDB supports storing binary data directly within BSON documents, however the size is limited by the database's maximum document size (16MB as of the 1.8 production version of MongoDB; 4MB in older versions).

GridFS works by splitting large files into small "chunks", usually 256k in size.

GridFS uses two collections to store data:

  • chunks contains the binary segments of files
  • files contains metadata including the filename, content type, md5 checksum, and any optional information added by the developer

Each file saved in GridFS will have one document in the files collection and one or more documents in the chunks collection.

The GridFS collections also use a prefix (aka namespace). The default prefix is fs, so the default collection names are fs.chunks and fs.files.

GridFS collections are stored in normal MongoDB databases, and can be scaled with standard features such as replication and sharding.

Working with GridFS

All of the mongodb.org supported drivers implement the GridFS spec.

The command-line utility mongofiles can also be used to save, retrieve, list, search, and remove files in GridFS.

Related Links

1101 questions
6
votes
2 answers

How to execute server side file handling operations in Meteor?

I'm storing Word(.docx) files using GridFS on the server. I'd like to be able to merge the documents into one Word file by using the docx-builder NPM package. Here's how I am uploading the files: Meteor.methods({ uploadFiles: function (files) { …
Sam Fischer
  • 1,442
  • 19
  • 35
6
votes
1 answer

Is Rails 3 removing Metal?

Is Rails 3 removing Metal? I am trying to implement Mongoid, Carrierwave and grid_fs. I have followed guide 1. However, I have read that Rails 3 is removing Metal ( http://github.com/rails/rails/commit/ed34652d1aca148fea61c5309c1bd5ff3a55abfa ). So…
Christian Fazzini
  • 19,613
  • 21
  • 110
  • 215
6
votes
0 answers

Store files under 16MB in MongoDB using GridFs is a good idea?

I know MongoDB documents only supports up to 16MB. If I want to store files bigger than this value, I should use GridFs. Lower than that, they suggest to use BSON instead. I have made a test, saving 33000 files with 10MB each, 16500 using BSON and…
Gabriel
  • 952
  • 10
  • 31
6
votes
2 answers

how to download a file saved in gridFS using nodeJS

I need to download a resume from GridFS, below is the code ive written to do it, but this seems to not give me a physical file for download, this is used to reading the contents. How can i download the file? exports.getFileById = function(req,…
Syed Faizan
  • 901
  • 3
  • 14
  • 28
6
votes
1 answer

python PIL image how to save image to a buffer so can be used later?

I have a png file which should be convert to jpg and save to gridfs , I use python's PIL lib to load the file and do the converting job, the problem is I want to store the converted image to a MongoDB Gridfs, in the saving procedure, I can't just…
armnotstrong
  • 8,605
  • 16
  • 65
  • 130
6
votes
0 answers

BSON::Binary::InvalidType "my file content" is not a valid binary type. Please use one of :generic, :function, :old, :uuid_old, :uuid, :md5, :user.):

I am trying to save files into gridfs using carrierwave mongoid in rails. I am unable to do this process due to this error: BSON::Binary::InvalidType ("my file content" is not a valid binary type. Please use one of :generic, :function, :old,…
Ratnakar Vanapalli
  • 81
  • 1
  • 7
6
votes
1 answer

GridFS + NodeJS Retrieve image from mongoDB

I have the following JSON from mongoDB with images but I don't know how to display it: fs.chunks: { "_id" : ObjectId("542e684a8a1cec178a172673"), "n" : 1, "data" :…
Jose
  • 1,159
  • 1
  • 9
  • 23
6
votes
2 answers

Updating GridFS metadata using Spring Data MongoDB

I want to update single field of metadata in GrifFs files collection. I read the documentation about Spring Data MongoDB but did not find any API for that. The only solution I have found so far is to use the Mongo API directly to delete the…
Sumit D
  • 401
  • 4
  • 15
6
votes
1 answer

storing files in mongodb using gridfs and mongoose

I have the following code gridfs.js has the following code that writes file bytes exports.putFile = function(path, name, options, fn) { var db; db = mongoose.connection.db; options = parse(options); options.metadata.filename =…
Mel
  • 245
  • 5
  • 15
6
votes
2 answers

Read file with node.js, mongoose, gridfs-stream

I am using mongoose and gridfs-stream to store and read files from mongodb. I am following the example here: https://github.com/aheckmann/gridfs-stream Writing files into db is working fine but I faced a problem to read files. What the mongodb looks…
LKS
  • 673
  • 2
  • 10
  • 21
6
votes
2 answers

I want the benefits of GridFS but mainly storing files under 16MB

I need to create a storage system for files that will mostly be under 16MB but I want the benefits of GridFS like versioning, custom metadata, easy backup (with mongodump), etc. I'd say maybe 10% of my files would be over 16MB so I can't rely on…
Josh Kim
  • 215
  • 4
  • 14
6
votes
1 answer

MongoDB GridFS "illegal chunk format' exception

I've been writing an app in Node.js that stores images in MongoDB's GridFS file system. I've uploaded images through the app, and the images appear to be stored correctly: $ mongofiles -v -d speaker-karaoke get howard-basement-100x115.jpg Tue Jul 17…
Howard M. Lewis Ship
  • 2,247
  • 15
  • 23
5
votes
1 answer

Multer GridFsStorage Dynamic Configuration for Multiple MongoDB Connections

So I'm using MongoDB Atlas, Mongoose, Multer and GridFsStorage to upload files to a single database on the cloud which works fine, but I want to be able to upload files to different databases, using different database connections dynamically. In…
Nikitas IO
  • 696
  • 5
  • 20
5
votes
2 answers

How to resize and image before saving it with gridfs?

i'm a beginner with mongoDB and gridfs. I have a code which save in the database an incoming image. But I'd like to resize the image before saving it. It tried to use sharp, but I don't really understand how it works. I tried the following code, but…
ATM
  • 160
  • 2
  • 10
5
votes
1 answer

MongoDB doesn't scale properly when adding new shard with collection already filled

My MongoDB sharded cluster ingestion performances don't scale up when adding a new shard. I have a small cluster setup with 1 mongos + 1 config replica set (3 nodes) + N shards replica sets (3 nodes each). Mongos is on a dedicated Kubernetes node,…
Federico Bonelli
  • 761
  • 1
  • 7
  • 23
1 2 3
…
73 74