If you’ve landed here, sorry to disappoint, but this repository hasn’t been updated in close to a decade and is full of holes, was never particularly well-written code in the first place, and should not be considered for new systems.
The good news is that there is a successor which is better (and also won a Student Radio Award!) so you can use that instead.
IRIS is the Interchangeable Radio Ingest System. It won a Gold award for Best Technical Achievement from the Student Radio Awards. And your station can use it too.
It has one purpose, and one purpose only: Act as a firewall between presenters and staff, trying to upload their music, and what should be a very well maintained and looked-after music library in a radio playout system. Of course, this playout system could be anything- the current targets for automatic import to playout systems are the Rivendell broadcast automation system, as well as a generic tagged file output (Broadcast WAVE Format/Linear PCM, untagged Linear PCM or high-as-possible MPEG Layer 3 with IDv2/1 tags, tagged FLAC or tagged AAC) which can be ingested to a playout system manually or via auto-import tools built into the playout system.
The goal is to make adhesion to guidelines for technical quality in radio and other broadcast systems easier, and to automate as much of the ingest process as is possible in most workflows.
IRIS is a Ruby on Rails web application, for ease of use for both presenters and systems administrators. Ruby is a cinch to learn if you’ve not used it before, and trivial to read- so even if you don’t know how to extend or adapt it, you can at least work out what it’s doing by having a look under the hood.
The system is designed to be easy to adapt and modify for various scenarios and needs. It features flexible authentication (via the cancan plugin and Devise), a light set of dependencies on external programs to do audio and metadata processing, and a state machine based process for handling incoming uploads.
Audio first uploaded is scanned for technical (bitrate, samplerate, format, etc) metadata and non-technical (artist, title, album, copyright, composer etc) metadata. This is then correlated with the MusicBrainz DB where possible to fill in the blanks (ISRCs particularly). Audio is then unpacked to WAV for processing.
It features quality (bitrate/samplerate) filtering and supports filtering of explicit lyrics through the MusixMatch service (commercial in some cases, but reasonably cheap and free for non-commercial uses). Filtered uploads can be reviewed and either accepted or rejected by an administrator.
Audio being introduced to the system can be archived to FLAC prior to processing, preserving an original copy of the file in case normalization needs to be reversed. Originals of uploads can also be kept for debugging failed uploads etc, with the idea being that you nuke the originals every few days and keep the FLACs to hand in a consistent format long-term, since storage is cheap these days.
Once audio is unpacked to WAV (via ffmpeg, so incoming files can be pretty much anything), it is normalized with an aim to not comply per-se but at least motion towards the EBU R128 standard for loudness levelling.
Gains are adjusted to bring R128 loudness into line with -23 LUFS, and loudness range can be adjusted (compression of dynamic range) in situ, reducing the amount of post-playout processing required to achieve consistent levels.
All jobs and processing can be distributed across multiple machines with shared storage using a simple database based queue system that also ensures first-in-first-out processing priority, failure logging and retries, and per-upload logging.
An EBU R128 0.4s momentary loudness waveform is generated and stored in the DB before, after compression (if a compressor is used) and after the entire normalization step, allowing quick and easy review of track loudness and normalization effects.
IRIS requires:
- Ruby (1.9.2 recommended; I usually do a from-source install from ruby-lang.org, saves headaches) and development headers (not needed if you did it from source)
- The bundler gem (sudo gem install bundler)
- sox, ecasound, ffmpeg, flac and exiftool (all available in Ubuntu’s repos, other distros may vary)
- The SC4 LADSPA compressor (available in swh-plugins in Ubuntu- may be omitted if you’re not using LRA adjustment)
- loudness from libebur128 (https://github.com/jiixyj/libebur128)
- nginx with the nginx upload module (http://www.grid.net.ru/nginx/upload.en.html)
- A database server (PostgreSQL highly recommended)
- libxml-dev, libxslt-dev, graphviz-dev, ffmpeg-dev libraries
- Optionally for bwf export, bwfmetaedit (http://bwfmetaedit.sourceforge.net)
- Optionally for Rivendell import, rdimport available on job workers
- Optionally for mp3 export, lame and id3tag
- Optionally for aac export, faac
… and that’s it. Yeah, there’s a fair few things there- however on most platforms the only things you need to manually grab and build are nginx and libebur128, both of which are straightforward. The rest can be pulled in by a package manger.
Once you’ve installed the prerequisites, just git clone the repo to somewhere, cd in, and run bundle install to get the required gems. Once that’s done, copy config/database.yml.example to config/database.yml, configure it. You’ll also need to adjust config/settings.yml – if you plan to update directly, copy this to config/settings.local.yml and update that instead.
Then you just rake db:migrate to set up the DB, rails server thin to start the application server (run that in a screen), fire up nginx with the config in doc/nginx.conf.example (or roll your own based on that), and grab another screen and run rake jobs:work. Bam! Now you can open http://localhost:81 in a browser, and upload stuff and see everything just work. In theory.
If you’ve worked with Rails before: It’s a bog-standard Rails 3 app with some audio cleverness added in via external programs and a nginx upload module requirement. Flavour to taste.
Deployments can be distributed if you’ve got a few boxes you want to run it across to share the load (r128-scanner and ecasound can use a lot of
Currently Rivendell is supported for automatic library import- you will need rdimport available on the job workers to take advantage of this at present (rdxport delegation may be usable in the future). Currently all job workers will require rdimport, as delayed_job can pass an ImportJob to any worker.
It should be possible (assuming shared storage on common paths) to make rdimport on some workers just a wrapper script that sshes across to another machine which has rdimport installed and runs the command there.
A Myriad TCP/IP remote import function was planned, but P Squared released an auto import tool with Myriad 3.6 which permits automatic import from the filesystem. This can be used with the tagged file output mode to get basic metadata across.
The file import mode will output a file (format one of linear PCM WAV, Broadcast Wave Format/linear PCM, MPEG Layer 3, AAC and FLAC), which can then be automatically imported (such as in Myriad’s case) or manually imported by a librarian. This mode also allows you to use IRIS with pretty much any use in mind- be it broadcast video, processing audio for playout as a DJ, or just to get all the levels about right for a few playlists so you can listen in the car without fiddling with the volume all the time.
IRIS is the successor to MACIS, a similar system originally deloped to interface with the Myriad playout system at Insanity Radio 1287AM (http://insanityradio.com). MACIS was slapped together in a couple of days and didn’t work very well due to bugs in Myriad’s TCP/IP interface and database. It didn’t do anything clever, either, so mostly just got more duplicates in faster, and since it didn’t log anything… well, I learned a lot from it.
IRIS was originally slapped together in about a week and works way, way better than MACIS ever did. MACIS was also site-specific; IRIS was developed to be used in any radio station or other environment where audio content needed importing, processing and normalizing as part of a content ingest chain. It is targeted at professional users, but is simple enough (and free) so may be used by anyone.
IRIS wouldn’t work without a whole boatload of Linux audio tools, so a huge collective thank-you to the Linux audio development community. The sheer number of awesome tools, utilities and communities out there astounds me constantly.
IRIS was written by James Harrison (http://talkunafraid.co.uk), Insanity’s Head of Technology from 2010 till 2011, and freelance broadcast engineer, webapp developer and all-round geek.
IRIS is licensed under the GNU GPL v3 or greater, except where otherwise noted (external libraries and such).