8000 GitHub - jokerlin/ladder: Selfhosted alternative to 12ft.io. and 1ft.io bypass paywalls with a proxy ladder and remove CORS headers from any URL
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
/ ladder Public
forked from everywall/ladder

Selfhosted alternative to 12ft.io. and 1ft.io bypass paywalls with a proxy ladder and remove CORS headers from any URL

License

GPL-3.0 license
0 stars 190 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

jokerlin/ladder

BranchesTags
 
 

Repository files navigation

Ladder

License go.mod Go version GitHub tag (with filter) GitHub (Pre-)Release Date GitHub Downloads all releases GitHub Build Status (with event)

Ladder is a http web proxy. This is a selfhosted version of 1ft.io and 12ft.io. It is inspired by 13ft.

Why

Freedom of information is an essential pillar of democracy and informed decision-making. While media organizations have legitimate financial interests, it is crucial to strike a balance between profitability and the public's right to access information. The proliferation of paywalls raises concerns about the erosion of this fundamental freedom, and it is imperative for society to find innovative ways to preserve access to vital information without compromising the sustainability of journalism. In a world where knowledge should be shared and not commodified, paywalls should be critically examined to ensure that they do not undermine the principles of an open and informed society.

Disclaimer: This project is intended for educational purposes only. The author does not endorse or encourage any unethical or illegal activity. Use this tool at your own risk.

How it works

sequenceDiagram
    client->>+ladder: GET
    ladder-->>ladder: apply RequestModifications
    ladder->>+website: GET
    website->>-ladder: 200 OK
    ladder-->>ladder: apply ResultModifications
    ladder->>-client: 200 OK
Loading

Features

  • Bypass Paywalls
  • Remove CORS headers from responses, assets, and images ...
  • Apply domain based ruleset/code to modify response / requested URL
  • Keep site browsable
  • API
  • Fetch RAW HTML
  • Custom User Agent
  • Custom X-Forwarded-For IP
  • Docker container (amd64, arm64)
  • Linux binary
  • Mac OS binary
  • Windows binary (untested)
  • Removes most of the ads (unexpected side effect ¯\_(ツ)_/¯ )
  • Basic Auth
  • Disable logs
  • No Tracking
  • Limit the proxy to a list of domains
  • Expose Ruleset to other ladders
  • Fetch from Google Cache
  • Optional TOR proxy
  • A key to share only one URL

Limitations

Some sites do not expose their content to search engines, which means that the proxy cannot access the content. A future version will try to fetch the content from Google Cache.

Certain sites may display missing images or encounter formatting issues. This can be attributed to the site's reliance on JavaScript or CSS for image and resource loading, which presents a limitation when accessed through this proxy. If you prefer a full experience, please consider buying a subscription for the site.

Installation

Warning: If your instance will be publicly accessible, make sure to enable Basic Auth. This will prevent unauthorized users from using your proxy. If you do not enable Basic Auth, anyone can use your proxy to browse nasty/illegal stuff. And you will be responsible for it.

Binary

  1. Download binary here
  2. Unpack and run the binary ./ladder -r https://t.ly/14PSf
  3. Open Browser (Default: http://localhost:8080)

Docker

docker run -p 8080:8080 -d --env RULESET=https://t.ly/14PSf --name ladder ghcr.io/everywall/ladder:latest

Docker Compose

curl https://raw.githubusercontent.com/everywall/ladder/main/docker-compose.yaml --output docker-compose.yaml
docker-compose up -d

Helm

See README.md in helm-chart sub-directory for more information.

Usage

Browser

  1. Open Browser (Default: http://localhost:8080)
  2. Enter URL
  3. Press Enter

Or direct by appending the URL to the end of the proxy URL: http://localhost:8080/https://www.example.com

Or create a bookmark with the following URL:

javascript:window.location.href="http://localhost:8080/"+location.href
< 8000 div class="markdown-heading" dir="auto">

API

curl -X GET "http://localhost:8080/api/https://www.example.com"

RAW

http://localhost:8080/raw/https://www.example.com

Running Ruleset

http://localhost:8080/ruleset

Configuration

Environment Variables

Variable Description Value
PORT Port to listen on 8080
PREFORK Spawn multiple server instances false
USER_AGENT User agent to emulate Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
X_FORWARDED_FOR IP forwarder address 66.249.66.1
USERPASS Enables Basic Auth, format admin:123456 ``
LOG_URLS Log fetched URL's true
DISABLE_FORM Disables URL Form Frontpage false
FORM_PATH Path to custom Form HTML ``
RULESET Path or URL to a ruleset file, accepts local directories https://raw.githubusercontent.com/everywall/ladder-rules/main/ruleset.yaml or /path/to/my/rules.yaml or /path/to/my/rules/
EXPOSE_RULESET Make your Ruleset available to other ladders true
ALLOWED_DOMAINS Comma separated list of allowed domains. Empty = no limitations ``
ALLOWED_DOMAINS_RULESET Allow Domains from Ruleset. false = no limitations false

ALLOWED_DOMAINS and ALLOWED_DOMAINS_RULESET are joined together. If both are empty, no limitations are applied.

Ruleset

It is possible to apply custom rules to modify the response or the requested URL. This can be used to remove unwanted or modify elements from the page. The ruleset is a YAML file, a directory with YAML Files, or an URL to a YAML file that contains a list of rules for each domain. These rules are loaded on startup.

There is a basic ruleset available in a separate repository ruleset.yaml. Feel free to add your own rules and create a pull request.

- domain: example.com          # Includes all subdomains
  domains:                     # Additional domains to apply the rule
    - www.example.de
    - www.beispiel.de
  headers:
    x-forwarded-for: none      # override X-Forwarded-For header or delete with none
    referer: none              # override Referer header or delete with none
    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
    content-security-policy: script-src 'self'; # override response header
    cookie: privacy=1
  regexRules:
    - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
      replace: <script $1 script="/https://www.example.com/$3"
  injections:
    - position: head # Position where to inject the code
      append: |      # possible keys: append, prepend, replace
        <script>
          window.localStorage.clear();
          console.log("test");
          alert("Hello!");
        </script>
- domain: www.anotherdomain.com # Domain where the rule applies
  paths:                        # Paths where the rule applies
    - /article
  googleCache: false            # Use Google Cache to fetch the content
  regexRules:                   # Regex rules to apply
    - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
      replace: <script $1 script="/https://www.example.com/$3"
  injections:
    - position: .left-content article .post-title # Position where to inject the code into DOM
      replace: | 
        <h1>My Custom Title</h1>
    - position: .left-content article # Position where to inject the code into DOM
      prepend: | 
        <h2>Subtitle</h2>
- domain: demo.com
  headers:
    content-security-policy: script-src 'self';
    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
  urlMods:              # Modify the URL
    query:              
      - key: amp        # (this will append ?amp=1 to the URL)
        value: 1 
    domain:             
      - match: www      # regex to match part of domain
        replace: amp    # (this would modify the domain from www.demo.de to amp.demo.de)
    path:               
      - match: ^        # regex to match part of path
        replace: /amp/  # (modify the url from https://www.demo.com/article/ to https://www.demo.de/amp/article/)

Development

To run a development server at http://localhost:8080:

echo "dev" > handlers/VERSION
RULESET="./ruleset.yaml" go run cmd/main.go

Optional: Live reloading development server with cosmtrek/air

Install air according to the installation instructions.

Run a development server at http://localhost:8080:

air # or the path to air if you haven't added a path alias to your .bashrc or .zshrc

This project uses pnpm to build a stylesheet with the Tailwind CSS classes. For local development, if you modify styles in form.html, run pnpm build to generate a new stylesheet.

About

Selfhosted alternative to 12ft.io. and 1ft.io bypass paywalls with a proxy ladder and remove CORS headers from any URL

Resources

Readme

License

GPL-3.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Go 83.7%
  • HTML 14.2%
  • Dockerfile 1.1%
  • Other 1.0%
0