A library written in Rust that handles sorting and serializing a simple model of Person records. A Person looks like this:
{
"last_name": "Brennan",
"first_name": "Tom",
"email": "tjb1982@gmail.com",
"favorite_color": "red",
"dob": "8/19/1982"
}There are two clients that use this library: a CLI interface, and a ReST interface.
The CLI is designed to work exclusively with CSV, ingesting a list of input files and outputting a list of records in CSV format to stdout.
It can read from stdin as well as a list of files you provide, with or without headers, and works with any char separator you provide.
For example, in the following, the CLI reads a list of three files and outputs the results sorted in the default order (i.e., by last_name, first_name, ...; ascending).
cli file1.csv file2.csv file3.csvUsing the -S flag, the separator is set to read CSV with a "|" for the separator, and using the -E flag set, the CLI will assume that all of the files have a header.
cli -S"|" -E -- file1.csv file2.csv file3.csvReading from stdin is just a matter of providing a "-" in place of one of the files. For example, if you want to generate some test records using the functional test suite, and have the CLI read those in as input, do:
python3 ./functional_tests/gen_people.py 500 | cli - file1.csv file2.csv file3.csvYou can also provide a mapping of separator/has-header combinations for each input file using flags.
cli -s"|" -e true \
-s"," -e false \
-s" " -e true \
-- file1.csv file2.csv file3.csvThe output can also contain a header:
cli file1.csv file2.csv file3.csv -tSorting is available by a sequence of flags. For example, to sort by favorite_color ascending, then first_name descending:
cli file1.csv file2.csv file3.csv -f favorite_color -f first_name -d descYou can also discover what fields there are using -a.
The API is a ReST API with the following endpoints:
POST /records- post a single data line of eithertext/csvorapplication/jsonGET /records/:field_name- returns records sorted by:field_nameGET /records/color- alias for/records/favorite_colorGET /records/birthdate- alias for/records/dobGET /records/name- alias for/records/last_name
The listing endpoints implement a basic pagination scheme of page and per-page with resultset.
These endpoints also handle a direction query param, to indicate what sort direction, asc or desc (asc is the default).
curl "http://localhost:8082/records/color?direction=desc&page=5&per-page=5"
The POST /records endpoint takes either text/csv or application/json in the Content-Type header.
For text/csv, the body should contain a single line of CSV. The separator should be ',' and there should be no header:
Brennan, Tom, tjb1982@gmail.com, red, 8/19/1982
N.B., space around all values will be trimmed.
For application/json, the body should contain an Object with key/value pairs:
{
"last_name": "Brennan",
"first_name": "Tom",
"email": "tjb1982@gmail.com",
"favorite_color": "red",
"dob": "8/19/1982"
}To run both the CLI and ReST API with minimal effort, a Dockerfile is provided that builds the repository and launches the API service on port 8082 with a pre-populated 7095 database of 1000 randomly generated records.
docker build -t homework .
docker run --name homework -p 8082:8082 homeworkTo run the functional test suite, log into the running container:
docker exec -it homework bashOnce logged in, you have access to the target binaries and python, etc. Run the test suite like:
python3 functional_tests/main.py cli | json_pp | lessN.B., the test suite reports in JSON format, so piping to
json_ppandlessmakes it easier to view the results.
The test suite also takes a couple of arguments which you can see by passing the -h or --help flag.
Although the requirements for the homework assignment were pretty small, and could easily have been done in a dynamic language (such as Python or Clojure), I chose to do it in Rust for a few reasons:
I wanted to take this as an opportunity to learn a new language
Setting up a non-trivial proof-of-concept in an unfamiliar technology is a relatively common on-the-job exercise in the real world. And Rust is a seriously difficult language to learn. One of the best ways I know how to "put my best foot forward" is to show that I can go from zero to "productive" relatively quickly in pretty much any language.
The key word here is "productive," as opposed to "mastered." Rust has a lot of complex concepts around its ownership model, and it will take some time before I feel confidently creative with it, the way I do with, say, JavaScript, Python, or Clojure. Or even C/C++.
The Rust ecosystem may not be as mature as the JVM, or Python, ecosystems, etc., but the tools and libraries it does have are particularly well-suited to creating nice-looking, safe/correct, CLIs and microservices. What makes Rust particularly good for APIs is that it compiles to a static binary and doesn't need garbage collection, like C++; but unlike C++ is much better at producing memory-safe code without sacrificing performance.