Unlurker helps you find the liveliest discussions on news.ycombinator.com so you can jump in before discussion dies.
The site hn.unlurker.com is built on this package along with a thin API and front-end.
This repo contains a robust client library for the HN API.
It also publishes two tools based on the library:
-
unlis a command-line version of hn.unlurker.com. -
hnenables retrieval of raw JSON data from the HN API
Install on Linux or Mac OS to /usr/local/bin (requires curl, tar, and sudo).
curl -Ls "https://github.com/jasonthorsness/unlurker/releases/latest/download/unl_\
$(uname -s | tr '[:upper:]' '[:lower:]')_\
$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')\
.tar.gz" \
| sudo tar -xz -C /usr/local/bin
unl --limit 1curl -Ls "https://github.com/jasonthorsness/unlurker/releases/latest/download/hn_\
$(uname -s | tr '[:upper:]' '[:lower:]')_\
$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')\
.tar.gz" \
| sudo tar -xz -C /usr/local/bin
hn new --limit 1unl
# Show the 3 latest stories with active discussions
unl --limit 3
# Show stories with activity from at least 10 unique users in the last hour max 24 hours old
unl --min-by 10 --window 1h --max-age 24hhn
# Find stories from the 'new' list with 'rust' in the title
hn new | grep rust
# Download the latest 10000 records to out.json (with resume support)
hn scan -l10000 -c- -o out.jsonBoth tools have some common flags related to persistent caching.
| flag | purpose |
|---|---|
| --cache-path string | override the default persistent cache path |
| --no-cache | disable persistent caching |
Since most stories and comments rarely change, both tools maintain a shared persistent cache of retrieved content. Items will be retrieved from the cache until deemed stale. How long it takes for an item to be considered stale depends on the cached item's age, starting at one minute and reaching immutable for items older than a couple of weeks.
This persistent cache file defaults to hn.db stored in the user-specific cache or global temp
directory. To see the default storage location for your machine, just run the tool with --help and
note the default for --cache-path.
To disable this persistent caching, use --no-cache. To change the location use --cache-path.
unl finds active discussions on news.ycombinator.com
Usage:
unl [flags]
Examples:
unl --max-age 8h --window 30m --min-by 3 --limit 3
Flags:
--cache-path string cache file path (default "/home/jason/.cache/hn.db")
-h, --help help for unl
-l, --limit int limit the number of results
--max-age duration maximum age for items (default 24h0m0s)
--min-by int minimum count of unique contributors to activity (default 3)
--no-cache disable cache
--no-color disable color
--window duration time window for activity (default 1h0m0s)
unl works best in wide terminals because it doesn't wrap text. The sample output below is 100
characters. It shows the full links because most terminals make them clickable and navigating to the
discussions is the point of the tool.
https://news.ycombinator.com/item?id=43740065 schappim 2h13m Ask HN: What Did You Learn Too …
https://news.ycombinator.com/item?id=43740647 hiAndrewQuinn 15m |\- Everyone else is giving vag…
https://news.ycombinator.com/item?id=43740589 WheelsAtLarge 26m |\- You need to make sure manag…
https://news.ycombinator.com/item?id=43740601 mindcrime 23m | \- To add to that: there may …
https://news.ycombinator.com/item?id=43740586 mindcrime 27m \- First of all, I don't think…
hn retrieves data from the HN API (https://github.com/HackerNews/API)
Usage:
hn [command] [flags]
hn [command]
Examples:
hn new --limit 3
hn user jasonthorsness --submitted --limit 5
hn scan --limit 10000 --continue-at - -o out.json
Available Commands:
best Retrieve items from the best list
completion Generate the autocompletion script for the specified shell
help Help about any command
new Retrieve items from the new list
scan Retrieve a range of items from the HN API
top Retrieve items from the top list
user Retrieve a user's profile or their submitted items
Flags:
--cache-path string cache file path (default "/home/jason/.cache/hn.db")
-h, --help help for hn
--max-connections int maximum TCP connections to open (default 100)
7AA8
--no-cache disable caching
-o, --output string output filename
Use "hn [command] --help" for more information about a command.
hn outputs JSON exactly as the HN API returns it. If you want to filter or pretty-print the JSON,
use a tool like jq.
{"by":"leonewton253","descendants":1,"id":43740739,"kids":[43740740],"score":1,"time":1745110876,"title":"SteamOS: Nix Edition. First Beta Release","type":"story","url":"https://github.com/SteamNix/SteamNix"}
The scan command can be used to download the entire HN database. Since this can take quite some
time, make sure you use the --continue-at - option with an output file -o out.json. If you do
this, and something goes wrong (or you simply CTRL+C for some reason), you will be able to
re-execute the same command and it will look at the contents of out.json to figure out where to
correctly resume. You can resume with a different limit and cache settings.
If you use scan --asc you can keep appending new items to the file by re-running the command.
Since recent items often change, you might want to trim the last few lines from the file in case
they have changed. This bash script can accomplish the task:
input=input.json trim=1000 && \
truncate -s -$(tail -n "$trim" "$input" | wc -c) "$input" && \
hn scan --no-cache --asc -c- -o "$input"You'll need to be using at least go 1.24.3.
go get github.com/jasonthorsness/unlurker/hnFor a simple examples of using the client library refer to cmd/unl/main.go and API.
This project requires the go 1.24.3 SDK. Run 'make' to build both tools.