jvalue · felix-oq · Jan 13, 2023 · Jan 12, 2023 · Jan 12, 2023 · Jan 12, 2023
diff --git a/README.md b/README.md
@@ -2,11 +2,16 @@
 
 ## Projects overview
 
-| Name                                                      | Description                                                   |
-| --------------------------------------------------------- | ------------------------------------------------------------- |
-| [`language-server`](./libs/language-server/README.md)     | Jayvee language definition and language server implementation |
-| [`vs-code-extension`](./apps/vs-code-extension/README.md) | Visual Studio Code extension for the language                 |
-| [`interpreter`](./apps/interpreter/README.md)             | Command line tool for interpreting `.jv` files                |
+| Name                                              | Description                                                    |
+|---------------------------------------------------|----------------------------------------------------------------|
+| [`language-server`](./libs/language-server)       | Jayvee language definition and language server implementation  |
+| [`interpreter`](./apps/interpreter)               | Command line tool for interpreting Jayvee files                |
+| [`vs-code-extension`](./apps/vs-code-extension)   | Visual Studio Code extension for editing Jayvee files          |
+| [`monaco-editor`](./libs/monaco-editor)           | React component for editing Jayvee files                       |
+| [`execution`](./libs/execution)                   | Shared code for Jayvee extensions and the interpreter          |
+| [`extensions/std`](./libs/extensions/std)         | Standard Jayvee extension consisting of the extensions below   |
+| [`extensions/rdbms`](./libs/extensions/rdbms)     | Jayvee extension for relational databases                      |
+| [`extensions/tabular`](./libs/extensions/tabular) | Jayvee extension for tabular data                              |
 
 ## Quick start
 

diff --git a/docs/jayvee-extensions.md b/docs/jayvee-extensions.md
@@ -0,0 +1,261 @@
+# Jayvee extensions
+
+## Concepts
+
+### Jayvee extension
+
+A Jayvee extension is used to add new block types to the language without having to modify the actual grammar of the language. Such a Jayvee extension usually consists of two parts: a language extension and an execution extension which are described in the upcoming two sections.
+
+Jayvee extensions that shall be used by default are bundled into the so-called [standard extension](../libs/extensions/std). That way, the [language server](../libs/language-server/) and the [interpreter](../apps/interpreter/) are able to load them out of the box.
+
+### Language extension
+
+A language extension defines meta information of block types which are required by the [language server](../libs/language-server/). Such meta information describes properties of block types such as their names, input / output types and their attributes.
+
+Note that language extensions don't define any behavior. Instead, this is done by the corresponding execution extension.
+
+### Execution extension
+
+An execution extension defines behavior for block types. They build on the meta information from the corresponding language extension, e.g. input / output types of the block need to match the signature of the execution method and attributes are accessed by their specified name.
+
+Execution extensions are only required by the [interpreter](../apps/interpreter/) and not necessarily by the [language server](../libs/language-server/) as they solely define behavior.
+
+## Recipes
+
+### Add a new Jayvee extension
+
+#### 1. Generate language and execution libraries
+
+```bash
+npx nx g @nrwl/node:library --name="extensions/<extension-name>/lang"
+npx nx g @nrwl/node:library --name="extensions/<extension-name>/exec"
+```
+
+#### 2. Create extension classes
+
+In `libs/extensions/<extension-name>/lang/src/extension.ts`:
+
+```typescript
+import {
+  BlockMetaInformation,
+  JayveeLangExtension,
+} from '@jayvee/language-server';
+
+export class MyLangExtension implements JayveeLangExtension {
+  getBlockMetaInf(): BlockMetaInformation[] {
+    return [];
+  }
+}
+```
+
+In `libs/extensions/<extension-name>/exec/src/extension.ts`:
+
+```typescript
+import {
+  BlockExecutor,
+  BlockExecutorType,
+  JayveeExecExtension,
+} from '@jayvee/execution';
+
+export class MyExecExtension implements JayveeExecExtension {
+  getBlockExecutors(): Array<
+    BlockExecutorType<BlockExecutor<unknown, unknown>>
+  > {
+    return [];
+  }
+}
+```
+
+#### 3. Export extension classes
+
+In `libs/extensions/<extension-name>/lang/src/index.ts`:
+
+```typescript
+export * from './extension';
+```
+
+In `libs/extensions/<extension-name>/exec/src/index.ts`:
+
+```typescript
+export * from './extension';
+```
+
+#### 4. Register new extension classes in the standard extension
+
+In `libs/extensions/std/lang/src/extension.ts`:
+
+```typescript
+// ...
+
+import { MyLangExtension } from '@jayvee/extensions/<extension-name>/lang';
+
+export class StdLangExtension implements JayveeLangExtension {
+  private readonly wrappedExtensions: JayveeLangExtension[] = [
+    // ...
+    // Register your language extension here:
+    new MyLangExtension(),
+    // ...
+  ];
+
+  // ...
+}
+```
+
+In `libs/extensions/std/exec/src/extension.ts`:
+
+```typescript
+// ...
+
+import { MyExecExtension } from '@jayvee/extensions/<extension-name>/exec';
+
+export class StdExecExtension implements JayveeExecExtension {
+  private readonly wrappedExtensions: JayveeExecExtension[] = [
+    // ...
+    // Register your execution extension here:
+    new MyExecExtension(),
+    // ...
+  ];
+
+  // ...
+}
+```
+
+### Add a new block type in a Jayvee extension
+
+#### 1. Implement `BlockMetaInformation`
+
+The following example defines a block type called `MyExtractor`. This block type, for instance, takes no input and outputs a sheet. Furthermore, it defines two attributes:
+
+- `url` of type string
+- `limit` of type integer and default value `10`
+  - Considered optional due to the specified default value
+
+In `libs/extensions/<extension-name>/lang/src/lib/my-extractor-meta-inf.ts`:
+
+```typescript
+import {
+  AttributeType,
+  BlockMetaInformation,
+  SHEET_TYPE,
+  UNDEFINED_TYPE,
+} from '@jayvee/language-server';
+
+export class MyExtractorMetaInformation extends BlockMetaInformation {
+  constructor() {
+    super(
+      // How the block type should be called:
+      'MyExtractor',
+
+      // Input type:
+      UNDEFINED_TYPE,
+
+      // Output type:
+      SHEET_TYPE,
+
+      // Attribute definitions:
+      {
+        url: {
+          type: AttributeType.STRING,
+        },
+        limit: {
+          type: AttributeType.INT,
+          defaultValue: 10,
+        },
+      }
+    );
+  }
+}
+```
+
+> **Note**
+> Use `UNDEFINED_TYPE` whenever you expect no input or output:
+>
+> - Use it as input type if you want to define an extractor
+> - Use it as output type if you want to define a loader
+
+#### 2. Register the new `BlockMetaInformation` in the language extension
+
+In `libs/extensions/<extension-name>/lang/src/extension.ts`:
+
+```typescript
+// ...
+
+import { MyExtractorMetaInformation } from './lib/my-extractor-meta-inf';
+
+export class MyLangExtension implements JayveeLangExtension {
+  getBlockMetaInf(): BlockMetaInformation[] {
+    return [
+      // ...
+      // Register your meta information here:
+      new MyExtractorMetaInformation(),
+      // ...
+    ];
+  }
+}
+```
+
+#### 3. Implement `BlockExecutor`
+warn
+The following example implements an executor for the previously defined block type `MyExtractor`.
+
+The `execute` method defines the behavior when a block is executed. Its signature matches the input and output types defined in `MyExtractorMetaInformation`.
+
+In `libs/extensions/<extension-name>/exec/src/lib/my-extractor-executor.ts`:
+
+```typescript
+import * as R from '@jayvee/execution';
+import { BlockExecutor } from '@jayvee/execution';
+import { Sheet } from '@jayvee/language-server';
+
+export class MyExtractorExecutor extends BlockExecutor<void, Sheet> {
+  constructor() {
+    // Needs to match the name in meta information:
+    super('MyExtractor');
+  }
+
+  override async execute(): Promise<R.Result<Sheet>> {
+    // Accessing attribute values by their name:
+    const url = this.getStringAttributeValue('url');
+    const limit = this.getIntAttributeValue('limit');
+
+    // ...
+
+    if (error) {
+      return R.err(...);
+    }

+    return R.ok(...);
+  }
+}
+```
+
+> **Warning**
+> The generic types of `BlockExecutor<I,O>` need to match the input and output types of the corresponding `BlockMetaInformation`.
+>
+> - Use `undefined` or `void` for `UNDEFINED_TYPE`
+> - For other types, see [`libs/language-server/src/lib/types/io-types.ts`](../libs/language-server/src/lib/types/io-types.ts)
+>
+> Be careful, as there is currently no mechanism to detect such potential mismatches.
+
+#### 4. Register the new `BlockExecutor` in the execution extension
+
+In `libs/extensions/<extension-name>/exec/src/extension.ts`:
+
+```typescript
+// ...
+
+import { MyExtractorExecutor } from './lib/my-extractor-executor';
+
+export class MyExecExtension implements JayveeExecExtension {
+  getBlockExecutors(): Array<
+    BlockExecutorType<BlockExecutor<unknown, unknown>>
+  > {
+    return [
+      // ...
+      // Register your block executor here:
+      MyExtractorExecutor,
+      // ...
+    ];
+  }
+}
+```