- Updated cfs:grid mongodb dependency to 2.2.4 (Meteor 1.4)
CollectionFS is a suite of Meteor packages that together provide a complete file management solution including uploading, downloading, storage, synchronization, manipulation, and copying. It supports several storage adapters for saving to the local filesystem, GridFS, or S3, and additional storage adapters can be created.
Victor Leung wrote a great quick start guide for the most basic image uploading and displaying task.
Check out the Wiki for more information, code examples and how-tos.
Table of Contents generated with DocToc
- Important Notes
- Installation
- Introduction
- Getting Started
- Storage Adapters
- File Manipulation
- Image Manipulation
- Filtering
- An FS.File Instance
- Security
- Display an Uploaded Image
- UI Helpers
Due to a bug in the Cordova Android version that is used with Meteor 1.2, you will need to add the following to your mobile-config.js or you will have problems with this package on Android devices:
App.accessRule("blob:*");If you have Documentation feedback/requests please post on issue 206
Only Meteor 0.9.0 and later are currently supported
$ cd <app dir>You must add cfs:standard-packages, which is the main package:
$ meteor add cfs:standard-packagesYou must add at least one storage adapter package. See the Storage Adapters section for a list of the available storage adapter packages. At least cfs:gridfs or cfs:filesystem must be added, too, even if you are not using them. The temporary store requires one of them.
$ meteor add cfs:gridfs
# OR
$ meteor add cfs:filesystem
# OR
$ meteor add cfs:s3
# OR
$ meteor add cfs:dropbox
# OR
$ meteor add iyyang:cfs-aliyunDepending on what you need to do, you may need to add additional add-on packages. These are explained in the documentation sections to which they apply.
$ meteor add <CFS add-on package name>The CollectionFS package makes available two important global variables:
FS.File and FS.Collection.
- An
FS.Fileinstance wraps a file and its data on the client or server. It is similar to the browserFileobject (and can be created from aFileobject), but it has additional properties and methods. Many of its methods are reactive when the instance is returned by a call tofindorfindOne. - An
FS.Collectionprovides a collection in which information about files can be stored. It is backed by an underlying normalMongo.Collectioninstance. Most collection methods, such asfindandinsertare available on theFS.Collectioninstance. If you need to call other collection methods such as_ensureIndex, you can call them directly on the underlyingMongo.Collectioninstance available throughmyFSCollection.files.
A document from a FS.Collection is represented as a FS.File.
CollectionFS also provides an HTTP upload package that has the necessary mechanisms to upload files, track upload progress reactively, and pause and resume uploads.
The first step in using this package is to define a FS.Collection.
common.js:
Images = new FS.Collection("images", {
stores: [new FS.Store.FileSystem("images", {path: "~/uploads"})]
});In this example, we've defined a FS.Collection named "images", which will
be a new collection in your MongoDB database with the name "cfs.images.filerecord". We've
also told it to use the filesystem storage adataper and store the files in ~/uploads on
the local filesystem. If you don't specify a path, a cfs/files folder in your app
container (bundle directory) will be used.
Your FS.Collection and FS.Store variables do not necessarily have to be global on the client or the server, but be sure to give them the same name (the first argument in each constructor) on both the client and the server.
To allow users to submit files to the FS Collection, you must create an allow rule in Server code:
server.js or within Meteor.isServer block:
Images.allow({
'insert': function () {
// add custom authentication code here
return true;
}
});Now we can upload a file from the client. Here is an example of doing so from the change event handler of an HTML file input:
Template.myForm.events({
'change .myFileInput': function(event, template) {
var files = event.target.files;
for (var i = 0, ln = files.length; i < ln; i++) {
Images.insert(files[i], function (err, fileObj) {
// Inserted new doc with ID fileObj._id, and kicked off the data upload using HTTP
});
}
}
});You can optionally make this code a bit cleaner by using a provided utility
method, FS.Utility.eachFile:
Template.myForm.events({
'change .myFileInput': function(event, template) {
FS.Utility.eachFile(event, function(file) {
Images.insert(file, function (err, fileObj) {
// Inserted new doc with ID fileObj._id, and kicked off the data upload using HTTP
});
});
}
});Notice that the only thing we're doing is passing the browser-provided File
object to Images.insert(). This will create a FS.File from the
File, link it with the Images FS.Collection, and then immediately
begin uploading the data to the server with reactive progress updates.
The insert method can directly accept a variety of different file
representations as its first argument:
Fileobject (client only)Blobobject (client only)Uint8ArrayArrayBufferBuffer(server only)- A full URL that begins with "http:" or "https:"
- A local filepath (server only)
- A data URI string
Where possible, streams are used, so in general you should avoid using any of the buffer/binary options unless you have no choice, perhaps because you are generating small files in memory.
The most common usage is to pass a File object on the client or a URL on
either the client or the server. Note that when you pass a URL on the client,
the actual data download from that URL happens on the server, so you don't
need to worry about CORS. In fact, we recommend doing all inserts on the
client (managing security through allow/deny), unless you are generating
the data on the server.
When you need to insert a file that's located on a client, always call
myFSCollection.insert on the client. While you could define your own method,
pass it the fsFile, and call myFSCollection.insert on the server, the
difficulty is with getting the data from the client to the server. When you
pass the fsFile to your method, only the file info is sent and not the data.
By contrast, when you do the insert directly on the client, it automatically chunks the file's data after insert, and then queues it to be sent chunk by chunk to the server. And then there is the matter of recombining all those chunks on the server and stuffing the data back into the fsFile. So doing client-side inserts actually saves you all of this complex work, and that's why we recommend it.
Calling insert on the server should be done only when you have the file somewhere on the server filesystem already or you're generating the data on the server.
After the server receives the FS.File and all the corresponding binary file
data, it saves copies of the file in the stores that you specified.
If any storage adapters fail to save any of the copies in the designated store, the server will periodically retry saving them. After a configurable number of failed attempts at saving, the server will give up.
To configure the maximum number of save attempts, use the maxTries option
when creating your store. The default is 5.
Storage adapters handle retrieving the file data and removing the file data when you delete the file. There are currently four available storage adapters, which are in separate packages. Refer to the package documentation for usage instructions.
- cfs:gridfs: Allows you to save data to mongodb GridFS.
- cfs:filesystem: Allows you to save to the server filesystem.
- cfs:s3: Allows you to save to an Amazon S3 bucket.
- cfs:dropbox: Allows you to save to a Dropbox account.
- iyyang:cfs-aliyun: Allows you to save to Aliyun OSS Storage.
If you're using a storage adapter that requires sensitive information such as
access keys, we recommend supplying that information using environment variables.
If you instead decide to pass options to the storage adapter constructor,
then be sure that you do that only in the server code (and not simply within a
Meteor.isServer block).
You may want to manipulate files before saving them. For example, if a user uploads a large image, you may want to reduce its resolution, crop it, compress it, etc. before allowing the storage adapter to save it. You may also want to convert to another content type or change the filename or encrypt the file. You can do all of this by defining stream transformations on a store.
Note: Transforms only work on the server-side code
The most common type of transformation is a "write" transformation, that is,
a function that changes the data as it is initially stored. You can define
this function using the transformWrite option on any store constructor. If the
transformation requires a companion transformation when the data is later read
out of the store (such as encrypt/decrypt), you can define a transformRead
function as well.
For illustration purposes, here is an example of a transformWrite function that doesn't do anything:
transformWrite: function(fileObj, readStream, writeStream) {
readStream.pipe(writeStream);
}The important thing is that you must pipe the readStream to the writeStream before returning from the function. Generally you will manipulate the stream in some way before piping it.
Sometimes you also need to change a file's metadata before it is saved to a particular store. For example, you might have a transformWrite function that changes the file type, so you need a beforeWrite function that changes the extension and content type to match.
The simplest type of beforeWrite function will return an object with extension, name, or type properties. For example:
beforeWrite: function (fileObj) {
return {
extension: 'jpg',
type: 'image/jpg'
};
}This would change the extension and type for that particular store.
Since beforeWrite is passed the fileObj, you can optionally alter that directly. For example, the following would be the same as the previous example assuming the store name is "jpegs":
beforeWrite: function (fileObj) {
fileObj.extension('jpg', {store: "jpegs", save: false});
fileObj.type('image/jpg', {store: "jpegs", save: false});
}(It's best to provide the save: false option to any of the setters you call in beforeWrite.)
A common use for transformWrite is to manipulate images before saving them.
To get this set up:
- Install GraphicsMagick or ImageMagick on your development machine and on any server that will host your app. (The free Meteor deployment servers do not have either of these, so you can't deploy to there.) These are normal operating system applications, so you have to install them using the correct method for your OS. For example, on Mac OSX you can use
brew install graphicsmagickassuming you have Homebrew installed. - Add the
cfs:graphicsmagickMeteor package to your app:meteor add cfs:graphicsmagick
var createThumb = function(fileObj, readStream, writeStream) {
// Transform the image into a 10x10px thumbnail
gm(readStream, fileObj.name()).resize('10', '10').stream().pipe(writeStream);
};
Images = new FS.Collection("images", {
stores: [
new FS.Store.FileSystem("thumbs", { transformWrite: createThumb }),
new FS.Store.FileSystem("images"),
],
filter: {
allow: {
contentTypes: ['image/*'] //allow only images in this FS.Collection
}
}
});Check out the Wiki for more examples and How-tos.
- When you insert a file, a worker begins saving copies of it to all of the
stores you define for the collection. The copies are saved to stores in the
order you list them in the
storesoption array. Thus, you may want to prioritize certain stores by listing them first. For example, if you have an images collection with a thumbnail store and a large-size store, you may want to list the thumbnail store first to ensure that thumbnails appear on screen as soon as possible after inserting a new file. Or if you are storing audio files, you may want to prioritize a "sample" store over a "full-length" store.
You may specify filters to allow (or deny) only certain content types,
file extensions or file sizes in a FS.Collection with the filter option:
Images = new FS.Collection("images", {
filter: {
maxSize: 1048576, // in bytes
allow: {
contentTypes: ['image/*'],
extensions: ['png']
},
deny: {
contentTypes: ['image/*'],
extensions: ['png']
},
onInvalid: function (message) {
if (Meteor.isClient) {
alert(message);
} else {
console.log(message);
}
}
}
});Alternatively, you can pass your filters object to myFSCollection.filters().
To be secure, this must be added on the server. However, you should use the filter
option on the client, too, to help catch many of the disallowed uploads there
and allow you to display a helpful message with your onInvalid function.
You can mix and match filtering based on extension or content types. The contentTypes array also supports "image/*" and "audio/*" and "video/*" like the "accepts" attribute on the HTML5 file input element.
If a file extension or content type matches any of those listed in allow,
it is allowed. If not, it is denied. If it matches both allow and deny,
it is denied. Typically, you would use only allow or only deny,
but not both. If you do not pass the filter option, all files are allowed,
as long as they pass the tests in your FS.Collection.allow() and
FS.Collection.deny() functions.
The extension checks are used only when there is a filename. It's possible to upload a file with no name. Thus, you should generally use extension checks only in addition to content type checks, and not instead of content type checks.
The file extensions must be specified without a leading period. Extension matching is case-insensitive.
An FS.File instance is an object with properties similar to this:
{ _id: '', collectionName: '', // this property not stored in DB collection: collectionInstance, // this property not stored in DB createdByTransform: true, // this property not stored in DB data: data, // this property not stored in DB original: { name: '', size: 0, type: '', updatedAt: date }, copies: { storeName: { key: '', name: '', size: 0, type: '', createdAt: date, updatedAt: date } }, uploadedAt: date, anyUserDefinedProp: anything }