A cross-platform library for opening OS pipes, like those from
pipe on Linux or
CreatePipe
on Windows. The Rust standard library provides
Stdio::piped for
simple use cases involving child processes, but it doesn't support creating pipes directly.
This crate fills that gap. Update: Rust 1.87 added
std::io::pipe, so this crate is no longer
needed except to support older compiler versions.
When you work with pipes, you often end up debugging a deadlock at some point. These can be confusing if you don't know why they happen. Here are two things you need to know:
- Pipe reads block until some bytes are written or all writers are
closed. If you forget to close a writer, reads can block
forever. This includes writers inside a
std::process::Commandobject or writers given to child processes. - Pipes have an internal buffer of some fixed size. On Linux for example, pipe buffers are 64 KiB by default. Pipe writes block until buffer space is available or all readers are closed. If you have readers open but not reading, writes can block forever.
Deadlocked reads caused by a forgotten writer usually show up
immediately, which makes them relatively easy to fix once you know
what to look for. (See "Avoid a deadlock!" in the example code
below.) However, deadlocked writes caused by full pipe buffers are
trickier. These might only show up for larger inputs, and they might
be timing-dependent or platform-dependent. If you find that writing
to a pipe deadlocks sometimes, think about who's supposed to be
reading from that pipe and whether that thread or process might be
blocked on something else. For more on this, see the Gotchas
Doc
from the duct crate. (And
consider whether duct
might be a good fit for your use case.)
Here we write a single byte into a pipe and read it back out:
use std::io::prelude::*;
let (mut reader, mut writer) = os_pipe::pipe()?;
// XXX: If this write blocks, we'll never get to the read.
writer.write_all(b"x")?;
let mut output = [0];
reader.read_exact(&mut output)?;
assert_eq!(b"x", &output);This is a minimal working example, but as discussed in the section
above, reading and writing on the same thread like this is
deadlock-prone. If we wrote 100 KB instead of just one byte, this
example would block on write_all, it would never make it to
read_exact, and that would be a deadlock. Doing the read and write
from different threads or different processes would fix the
deadlock.
For a more complex example, here we join the stdout and stderr of a
child process into a single pipe. To do that we open a pipe, clone
its writer, and set that pair of writers as the child's stdout and
stderr. (This is possible because PipeWriter implements
Into<Stdio>.) Then we can read interleaved output from the pipe
reader. This example is deadlock-free, but note the comment about
closing the writers.
// We're going to spawn a child process that prints "foo" to stdout
// and "bar" to stderr, and we'll combine these into a single pipe.
let mut command = std::process::Command::new("python");
command.args(&["-c", r#"
import sys
sys.stdout.write("foo")
sys.stdout.flush()
sys.stderr.write("bar")
sys.stderr.flush()
"#]);
// Here's the interesting part. Open a pipe, clone its writer, and
// set that pair of writers as the child's stdout and stderr.
let (mut reader, writer) = os_pipe::pipe()?;
let writer_clone = writer.try_clone()?;
command.stdout(writer);
command.stderr(writer_clone);
// Now start the child process running.
let mut handle = command.spawn()?;
// Avoid a deadlock! This parent process is still holding open pipe
// writers inside the Command object, and we have to close those
// before we read. Here we do this by dropping the Command object.
drop(command);
// Finally we can read all the output and clean up the child.
let mut output = String::new();
reader.read_to_string(&mut output)?;
handle.wait()?;
assert_eq!(output, "foobar");Note that the duct crate
can reproduce the example above in a single line of code, with no
risk of deadlocks and no risk of leaking zombie
children.