| Env | Status |
|---|---|
| ✅ |
✅ Captures both screenshots and scrape to .txt and .json
✅ Set any type of User-Agent via config/useragent.txt
✅ Fast since it uses https://docs.astral.sh/uv/
Research tool to quickly scrape the 1st page of the github issues and prs from a given github repo. Its screenshots are taken and the headings of the issues and prs are stored as .txt and .json
- Download and install uv
curl -LsSf https://astral.sh/uv/install.sh | sh-
Download this repo
-
Run
setup.sh
- Bash script which will install required python libraries.
- Note this uses
playwright-pythonand will also install its dependencies.
- Enter the required repos to be scraped in the
config/repos.txtin the follwing format
org1/repo1
org2/repo2
.
.- If you want to use a specific user agent.
- Make a new file
config/useragent.txtwith the required user-agent or defaults will be used.
# Defaul user=agent
default_user_agent = "Mozilla/5.0 (Linux; Android 10; SM-G975F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.93 Mobile Safari/537.36"- Execute
uv run xo.py- A new
reportsdirectory will be created with each repo as its own directory containing its scraped assets
Some repositories may not be scraped properly if it uses pinned issues.