From 8e22c324120453167f36070f4b42df41cd2cec87 Mon Sep 17 00:00:00 2001
From: "dependabot-preview[bot]"
 <27856297+dependabot-preview[bot]@users.noreply.github.com>
Date: Tue, 31 Mar 2020 16:42:22 +0000
Subject: [PATCH 01/12] chore(deps): bump requests from 2.22.0 to 2.23.0

Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.23.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.23.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
---
 requirements.txt         | 2 +-
 requirements_minimum.txt | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/requirements.txt b/requirements.txt
index e8fabf7..901ff3e 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
 pymongo==3.10.1
 mysql-connector-python==8.0.19
-requests==2.22.0
+requests==2.23.0
 python-twitter==3.5
 websockets>=7.0,<8
diff --git a/requirements_minimum.txt b/requirements_minimum.txt
index e20605c..822be75 100644
--- a/requirements_minimum.txt
+++ b/requirements_minimum.txt
@@ -1 +1 @@
-requests==2.22.0
\ No newline at end of file
+requests==2.23.0
\ No newline at end of file

From f1bb1b853b6f3f882aae0966afcef2832c826af6 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 10 May 2020 17:53:04 +0200
Subject: [PATCH 02/12] docs(readme): update readme with note about pastebin
 api restrictions

---
 README.md | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 41ce8f3..169d8c3 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,10 @@
 
 **Please note:** This framework is **not** to be used for illegal actions. It can be used for querying public Pastebin pastes for e.g. your username or email address in order to increase your own security.
 
-### Setup pastepwn
+### ⚠️ Important note
+In April 2020 Pastebin [disabled access to their scraping API](https://twitter.com/rnd_infosec_guy/status/1248310762227093509) for a short period of time. At first people weren't able to access the scraping API in any way, but later on they re-enabled access to the API setup page. But since then it isn't possible to scrape "text" pastes. Only pastes with any kind of syntax set. That reduces the amount of pastes to a minimum, which reduced the usefulness of this tool.
+
+### Setting up pastepwn
 
 To use the pastepwn framework you need to follow these simple steps:
 

From 27b238f21e6dfa5da26badba25842f7710e964e1 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:17:05 +0200
Subject: [PATCH 03/12] docs: update mail domain

---
 pastepwn/__init__.py | 2 +-
 setup.py             | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/pastepwn/__init__.py b/pastepwn/__init__.py
index dd5abd4..54a5b49 100644
--- a/pastepwn/__init__.py
+++ b/pastepwn/__init__.py
@@ -8,6 +8,6 @@
 from .core.scrapinghandler import ScrapingHandler
 from .core.actionhandler import ActionHandler
 
-__author__ = "d-Rickyy-b (pastepwn@rickyy.de)"
+__author__ = "d-Rickyy-b (pastepwn@rico-j.de)"
 
 __all__ = ['PastePwn', 'Paste', 'PasteDispatcher', 'ScrapingHandler', 'ActionHandler']
diff --git a/setup.py b/setup.py
index 24d9088..5943ce9 100644
--- a/setup.py
+++ b/setup.py
@@ -47,7 +47,7 @@ def requirements():
       long_description_content_type='text/markdown',
       url='https://github.com/d-Rickyy-b/pastepwn',
       author='d-Rickyy-b',
-      author_email='pastepwn@rickyy.de',
+      author_email='pastepwn@rico-j.de',
       license='MIT',
       packages=packages,
       include_package_data=True,

From 320fe903c63ea80680a58bee3225bcc73fe3165e Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:17:36 +0200
Subject: [PATCH 04/12] docs: add readme for analyzer directory

---
 pastepwn/analyzers/README.md | 62 ++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 pastepwn/analyzers/README.md

diff --git a/pastepwn/analyzers/README.md b/pastepwn/analyzers/README.md
new file mode 100644
index 0000000..78cfa29
--- /dev/null
+++ b/pastepwn/analyzers/README.md
@@ -0,0 +1,62 @@
+# Analyzers
+This directory contains all the analyzers for pastepwn. An analyzer is a class that extends the [BasicAnalyzer]() class (or any other subclass of that) and overrides at least the `match` function.
+
+The match function gets passed a paste object. 
+The job of an analyzer is to check if a paste matches certain criteria. If it matches, pastepwn will execute the action(s), that is stored in the analyzer.
+
+
+## Example analyzers
+Analyzers can check for multiple things.
+A paste contains a certain, previously defined string.
+Or the paste matches a certain regex.
+Or the paste's syntax is set to `java`.
+And many other things.
+
+Check a few selected examples to get an idea:
+- [RegexAnalyzer](https://github.com/d-Rickyy-b/pastepwn/blob/master/pastepwn/analyzers/regexanalyzer.py) - Foundation for most other analyzers. Checks a
+ paste against a regex
+- [SteamKeyAnalyzer](https://github.com/d-Rickyy-b/pastepwn/blob/master/pastepwn/analyzers/steamkeyanalyzer.py) - Checks if a paste contains a Steam Key
+- [IBANAnalyzer](https://github.com/d-Rickyy-b/pastepwn/blob/master/pastepwn/analyzers/ibananalyzer.py) - Checks if a paste contains an IBAN
+
+
+## Create own analyzer
+Check out the implementations of a few analyzers and you'll get an idea on how to get started.
+
+```
+# -*- coding: utf-8 -*-
+from .basicanalyzer import BasicAnalyzer
+
+
+class MyAnalyzer(BasicAnalyzer):
+    name = "MyAnalyzer"
+
+    def __init__(self, actions, regex, flags=0, blacklist=None):
+        # We need co call the init of super, to initialize some settings in the basicanalyzer
+        super().__init__(actions, self.name)
+        
+        # We can do some custom setup stuff to initialize e.g. blacklists
+        self.blacklist = blacklist or []
+
+    def match(self, paste):
+        # Here our pastes get matched. We can access all fields of the paste object
+        paste_title = paste.title or ""
+        return self.regex.findall(paste_title)
+```
+ 
+## Combining analyzers
+To combine multiple analyzers and hence multiple conditions, you can use bitwise operators.
+Those bitwise operators act as a logical operators by creating a new `MergedAnalyzer` class that handles the individual analyzers internally. 
+
+`&` - bitwise AND for combining analyzers with a logical AND
+`|` - bitwise OR for combining analyzers with a logical OR
+
+```
+analyzer1 = SomeAnalyzer(...)
+analyzer2 = SomeOtherAnalyzer(...)
+analyzer3 = ThirdAnalyzer(...)
+
+realAnalyzer = (SomeAnalyzer & SomeOtherAnalyzer) | ThirdAnalyzer
+```
+
+The `realAnalyzer` only matches if either `analyzer1` and `analyzer2` both match, or if `analyzer3` matches.
+

From 475e465eda0272200b5a23be8c8506b16116c42f Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:17:54 +0200
Subject: [PATCH 05/12] docs: update inline documentation for MergedAnalyzer

---
 pastepwn/analyzers/basicanalyzer.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/pastepwn/analyzers/basicanalyzer.py b/pastepwn/analyzers/basicanalyzer.py
index 8993757..b6c3d53 100644
--- a/pastepwn/analyzers/basicanalyzer.py
+++ b/pastepwn/analyzers/basicanalyzer.py
@@ -66,7 +66,10 @@ def __repr__(self):
 
 
 class MergedAnalyzer(BasicAnalyzer):
-    """Basic analyzer class"""
+    """
+    Combination class to combine multiple analyzers into a single one
+    Doesn't need to be created manually - use the binary operators (& and |) to combine multiple analyzers.
+    """
     name = "MergedAnalyzer"
 
     def __init__(self, base_analyzer, and_analyzer=None, or_analyzer=None):

From 71dbf6ae11f52e144b263cb438dc07217342a98e Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:18:49 +0200
Subject: [PATCH 06/12] docs: add newline in analyzer readme

---
 pastepwn/analyzers/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pastepwn/analyzers/README.md b/pastepwn/analyzers/README.md
index 78cfa29..eda5d6b 100644
--- a/pastepwn/analyzers/README.md
+++ b/pastepwn/analyzers/README.md
@@ -47,7 +47,7 @@ class MyAnalyzer(BasicAnalyzer):
 To combine multiple analyzers and hence multiple conditions, you can use bitwise operators.
 Those bitwise operators act as a logical operators by creating a new `MergedAnalyzer` class that handles the individual analyzers internally. 
 
-`&` - bitwise AND for combining analyzers with a logical AND
+`&` - bitwise AND for combining analyzers with a logical AND  
 `|` - bitwise OR for combining analyzers with a logical OR
 
 ```

From d07fc5dca7e154e43d615bf9ee64c51232effd44 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:19:48 +0200
Subject: [PATCH 07/12] docs: fix analyzer naming in readme

---
 pastepwn/analyzers/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pastepwn/analyzers/README.md b/pastepwn/analyzers/README.md
index eda5d6b..01c72ae 100644
--- a/pastepwn/analyzers/README.md
+++ b/pastepwn/analyzers/README.md
@@ -55,7 +55,7 @@ analyzer1 = SomeAnalyzer(...)
 analyzer2 = SomeOtherAnalyzer(...)
 analyzer3 = ThirdAnalyzer(...)
 
-realAnalyzer = (SomeAnalyzer & SomeOtherAnalyzer) | ThirdAnalyzer
+realAnalyzer = (analyzer1 & analyzer2) | analyzer3
 ```
 
 The `realAnalyzer` only matches if either `analyzer1` and `analyzer2` both match, or if `analyzer3` matches.

From 7d13460f150d021c7d55eef7d7a2cb5a5a100dce Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:23:47 +0200
Subject: [PATCH 08/12] docs: update analyzer example in readme

---
 pastepwn/analyzers/README.md | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/pastepwn/analyzers/README.md b/pastepwn/analyzers/README.md
index 01c72ae..4a0dca6 100644
--- a/pastepwn/analyzers/README.md
+++ b/pastepwn/analyzers/README.md
@@ -22,7 +22,7 @@ Check a few selected examples to get an idea:
 ## Create own analyzer
 Check out the implementations of a few analyzers and you'll get an idea on how to get started.
 
-```
+```python
 # -*- coding: utf-8 -*-
 from .basicanalyzer import BasicAnalyzer
 
@@ -40,7 +40,9 @@ class MyAnalyzer(BasicAnalyzer):
     def match(self, paste):
         # Here our pastes get matched. We can access all fields of the paste object
         paste_title = paste.title or ""
-        return self.regex.findall(paste_title)
+
+        # We can for example check if the paste title contains a certain string
+        return "Secret" in paste_title
 ```
  
 ## Combining analyzers

From b5f75a3d31c6c63fd5046fde961c5fa693b5d8a3 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sun, 7 Jun 2020 17:24:18 +0200
Subject: [PATCH 09/12] docs: update code example syntax in readme

---
 pastepwn/analyzers/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pastepwn/analyzers/README.md b/pastepwn/analyzers/README.md
index 4a0dca6..622b5eb 100644
--- a/pastepwn/analyzers/README.md
+++ b/pastepwn/analyzers/README.md
@@ -52,7 +52,7 @@ Those bitwise operators act as a logical operators by creating a new `MergedAnal
 `&` - bitwise AND for combining analyzers with a logical AND  
 `|` - bitwise OR for combining analyzers with a logical OR
 
-```
+```python
 analyzer1 = SomeAnalyzer(...)
 analyzer2 = SomeOtherAnalyzer(...)
 analyzer3 = ThirdAnalyzer(...)

From 8f5f53100d36874549f1bda2add426c162a58f30 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Fri, 19 Jun 2020 23:41:30 +0200
Subject: [PATCH 10/12] fix: also recognize ipv6 error messages

Welp... I never actually got IPv6 error messages until recently, so I never thought about it being an issue in the first place. Also removed the length check, because it's not making things a lot faster anyway. Also tweaked the regex by capturing from the beginning of the string.
---
 CHANGELOG.md                                  |  2 +
 .../exceptions/ipnotregisterederror.py        |  1 +
 pastepwn/scraping/pastebin/pastebinscraper.py |  4 +-
 .../pastebin/tests/pastebinscraper_test.py    | 49 +++++++++++++++++++
 4 files changed, 54 insertions(+), 2 deletions(-)
 create mode 100644 pastepwn/scraping/pastebin/tests/pastebinscraper_test.py

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 22ba3bf..d66b75d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Fixed
+- The PastebinScraper could not recognize error messages with IPv6 addresses.
 
 ## [1.3.0] - 2020-03-03
 ### Added
diff --git a/pastepwn/scraping/pastebin/exceptions/ipnotregisterederror.py b/pastepwn/scraping/pastebin/exceptions/ipnotregisterederror.py
index ad260f2..d65adb5 100644
--- a/pastepwn/scraping/pastebin/exceptions/ipnotregisterederror.py
+++ b/pastepwn/scraping/pastebin/exceptions/ipnotregisterederror.py
@@ -3,6 +3,7 @@
 
 
 class IPNotRegisteredError(Exception):
+    """Exception class indicating that your IP is not witelisted on pastebin"""
 
     def __init__(self, body):
         ip = re.search("YOUR IP: (.*?) DOES NOT HAVE ACCESS", body).group(1)
diff --git a/pastepwn/scraping/pastebin/pastebinscraper.py b/pastepwn/scraping/pastebin/pastebinscraper.py
index 5fa372d..b97a2e1 100644
--- a/pastepwn/scraping/pastebin/pastebinscraper.py
+++ b/pastepwn/scraping/pastebin/pastebinscraper.py
@@ -37,9 +37,9 @@ def __init__(self, paste_queue=None, exception_event=None, api_hit_rate=None):
 
     def _check_error(self, body, key=None):
         """Checks if an error occurred and raises an exception if it did"""
-        pattern = r"YOUR IP: \d{1,3}.\d{1,3}.\d{1,3}.\d{1,3} DOES NOT HAVE ACCESS\.\s+VISIT: https:\/\/pastebin\.com\/doc_scraping_api TO GET ACCESS!"
+        pattern = r"^YOUR IP: ((\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})|((([0-9A-Fa-f]{1,4}:){7})([0-9A-Fa-f]{1,4})|(([0-9A-Fa-f]{1,4}:){1,6}:)(([0-9A-Fa-f]{1,4}:){0,4})([0-9A-Fa-f]{1,4}))) DOES NOT HAVE ACCESS\.\s+VISIT: https:\/\/pastebin\.com\/doc_scraping_api TO GET ACCESS!"
 
-        if 107 >= len(body) >= 99 and re.match(pattern, body):
+        if 131 >= len(body) and re.match(pattern, body):
             self._exception_event.set()
             raise IPNotRegisteredError(body)
 
diff --git a/pastepwn/scraping/pastebin/tests/pastebinscraper_test.py b/pastepwn/scraping/pastebin/tests/pastebinscraper_test.py
new file mode 100644
index 0000000..4b214e6
--- /dev/null
+++ b/pastepwn/scraping/pastebin/tests/pastebinscraper_test.py
@@ -0,0 +1,49 @@
+# -*- coding: utf-8 -*-
+import unittest
+from pastepwn.scraping.pastebin import PastebinScraper
+from pastepwn.scraping.pastebin.exceptions import IPNotRegisteredError, PasteDeletedException, PasteNotReadyException, PasteEmptyException
+
+
+class TestPastebinscraper(unittest.TestCase):
+
+    def setUp(self) -> None:
+        self.pastebinscraper = PastebinScraper()
+
+    def test_empty(self):
+        with self.assertRaises(PasteEmptyException):
+            self.pastebinscraper._check_error("")
+
+    def test_not_ready(self):
+        with self.assertRaises(PasteNotReadyException):
+            self.pastebinscraper._check_error("File is not ready for scraping yet. Try again in 1 minute.")
+
+    def test_deleted(self):
+        with self.assertRaises(PasteDeletedException):
+            self.pastebinscraper._check_error("Error, we cannot find this paste.")
+
+    def _check_ip_not_registered(self, ip_list):
+        shell = "YOUR IP: {} DOES NOT HAVE ACCESS. VISIT: https://pastebin.com/doc_scraping_api TO GET ACCESS!"
+        for ip in ip_list:
+            with self.assertRaises(IPNotRegisteredError):
+                self.pastebinscraper._check_error(shell.format(ip))
+                print("The following IP was not detected: {}".format(ip))
+
+    def test_ipv4_not_registered(self):
+        """Test if the _check_error method detects different IPv4 addresses. It's okay to also detect invalid addresses where an octed is > 255)"""
+        ipv4_test = ["1.1.1.1", "10.1.5.6", "1.10.5.6", "1.1.50.6", "1.1.5.60", "1.1.50.60", "1.10.50.60", "10.10.50.60", "10.10.50.255", "10.10.255.255",
+                     "10.255.255.255", "255.255.255.255", "333.333.333.333"]
+
+        self._check_ip_not_registered(ipv4_test)
+
+    def test_ipv6_not_registered(self):
+        ipv6_test = ["fe80::21d8:f50:c295:c4be", "2001:cdba:0000:0000:0000:0000:3257:9652", "2001:cdba:0:0:0:0:3257:9652", "2001:cdba::3257:9652",
+                     "2001:cdba::1222", "21DA:D3:0:2F3B:2AA:FF:FE28:9C5A", "2001:cdba::1:2:3:3257:9652", "FE80::8329", "FE80::FFFF:8329",
+                     "FE80::B3FF:FFFF:8329", "FE80::0202:B3FF:FFFF:8329", "FE80:0000:0000:0000:0202:B3FF:FFFF:8329"]
+        # TODO: IPv6 addresses with double colon AND full zero groups (of 16 bits) are currently not recognized by the used regex. An example address would
+        #  be: `FE80::0000:0000:0202:B3FF:FFFF:8329`
+
+        self._check_ip_not_registered(ipv6_test)
+
+
+if __name__ == '__main__':
+    unittest.main()

From 91915b216adc58a703f9e94fb34d69a13d42a5d4 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Fri, 19 Jun 2020 23:42:09 +0200
Subject: [PATCH 11/12] docs: fix typo

---
 pastepwn/core/pastepwn.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pastepwn/core/pastepwn.py b/pastepwn/core/pastepwn.py
index 9e9efcf..7b0aec2 100644
--- a/pastepwn/core/pastepwn.py
+++ b/pastepwn/core/pastepwn.py
@@ -71,7 +71,7 @@ def add_scraper(self, scraper, restart_scraping=False):
     def add_analyzer(self, analyzer):
         """
         Adds a new analyzer to the list of analyzers
-        :param analyzer: Instance of an BasicAnalyzer
+        :param analyzer: Instance of a BasicAnalyzer
         :return: None
         """
 

From b810c441645eac58c024a1d2df11762ea1320e05 Mon Sep 17 00:00:00 2001
From: Rico <d-Rickyy-b@users.noreply.github.com>
Date: Sat, 20 Jun 2020 01:24:12 +0200
Subject: [PATCH 12/12] docs: update changelog to 1.3.1

---
 CHANGELOG.md | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d66b75d..7c3775b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,9 +6,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+
+## [1.3.1] - 2020-06-20
 ### Fixed
 - The PastebinScraper could not recognize error messages with IPv6 addresses.
 
+### Docs
+- Started adding some readme files for the subfolders to explain certain parts of the code better 
+
 ## [1.3.0] - 2020-03-03
 ### Added
 - Implemented base64analyzer, which matches if a found base64 string decodes to valid ascii ([b535781](https://github.com/d-Rickyy-b/pastepwn/commit/b535781d7e1760c7f846432d7ef87b97784e2d49))
@@ -160,7 +165,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [1.0.8] - 2018-10-22
 First stable release
 
-[unreleased]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.3.0...HEAD
+[unreleased]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.3.1...HEAD
+[1.3.1]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.3.0...v1.3.1
 [1.3.0]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/d-Rickyy-b/pastepwn/compare/v1.0.16...v1.1.0