# Install from PyPI:
python3 -m pip install memoria_search
# Or from source code:
python3 -m pip install .

# Run:
python3 -m memoria.web --port 80

# Or run from source code without installing (you may need to install some dependencies):
PYTHONPATH=./src python -m memoria.web --port 80

Notes:

• Your distribution may require that you create a virtual environment to install Python packages.
• Memoria is currently designed to run under Python 3.12. Your mileage may vary attempting to run under Python 3.11.

Self-contained Compose (including an Elasticsearch instance):

# With Docker Compose or Podman Compose:
podman-compose --profile elasticsearch up

# Cleanup:
podman-compose down --volumes

Single Docker container (for use with an existing Elasticsearch instance):

# Build or pull
podman build -t ghcr.io/sidneys1/memoria .
podman pull ghcr.io/sidneys1/memoria

# With plain Docker or Podman
podman run --name memoria -e MEMORIA_ELASTIC_HOST=http://hostname:9200/ -p 80 ghcr.io/sidneys1/memoria

# Cleanup:
podman container rm memoria
podman image rm ghcr.io/sidneys1/memoria

Note that Podman commands may require sudo to run, or that you configure your Podman environment to run rootless.

You can deploy Memoria as a container. The provided Containerfile builds a lightweight image based on python:3.12-alpine, which runs Memoria under Uvicorn on the exposed port 80.

podman build -t sidneys1/memoria .

You can also deploy Memoria with Docker Compose or Podman Compose (as shown here).

The file compose.yaml shows the most basic Compose strategy, building and launching a Memoria container. You can use Memoria with an existing Elasticsearch instance like so¹:

# You may want to use the `memoria_elastic_password` secret by uncommenting the
# relevant sections of `compose.yaml` and running:
printf 'my-password-here' | podman secret create memoria_elastic_password -

export ELASTIC_HOST=http://hostname:9200/
podman-compose up --build 

A Compose profile named elasticsearch is also provided that will additionally launch an Elasticsearch container.

# To start self-contained. See notes below regarding default credentials.
podman-compose up --build --profile elasticsearch

Any of these settings can be configured with uppercase environment variables prefixed with MEMORIA_ (e.g., MEMORIA_ELASTIC_PASSWORD). Additionally, settings can be read from files from /run/secrets³, which will take precedence over any environment variables. For example, to set elastic_password with a Docker or Podman secret, you can:

printf 'my-password-here' | podman secret create memoria_elastic_password -
podman run --name memoria --secret memoria_elastic_password -p 80 sidneys1/memoria

There are currently three types of Memoria Plugins used during web content retrieval and processing:

• Downloaders
  Downloaders are responsible for accessing a URL and retrieving its content from the internet. They can provide this content in many different formats to the next plugin in the stack. The most basic Downloaders (like the built-in default, AiohttpDownloader) only support downloading raw HTML to provide to the remaining plugins.

• Filters
  Filters transform input from the previous plugin in the stack (either the Downloader or another Filter). They can change the content format or modify it in place.

  By default Memoria uses the built in HtmlContentFinder plugin to remove extraneous HTML elements and prune the input to a single <main>, <article>, or <... id="content"> element (if one exists).

• Extractors
  Extractors are the last plugin to run, and are responsible for converting the input from the previous plugin (either the Downloader or the last Filter) into plain text that will be stored in Elasticsearch for indexing and searching.

  By default Memoria uses the built in HtmlExtractor plugin to convert the input HTML into plain text. It also searches the original downloaded HTML (before any potential modification by Filter plugins) for <meta ...> values that could be used to enrich the Elasticsearch document, such as "author" or "description".

Other types of plugins:

• Scraping Rule Filters
  Scraping rule filter plugins allow the Scraping Rules in the Settings UI to be extended with new functionality. These filters help determine which history URLs will be retrieved and scraped.