8000 Playwright fail annotation by olexandr13 · Pull Request #481 · testomatio/reporter · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Playwright fail annotation #481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

8000
Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
lib/
2 changes: 1 addition & 1 deletion .eslintrc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ module.exports = {
'no-empty-function': 0,
camelcase: 0,
},
}
};
9 changes: 5 additions & 4 deletions .github/workflows/linter-and-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ concurrency:

on:
push:
branches: [master]
branches: [master, 2.x]
pull_request:
branches: [master]
branches: [master, 2.x]

jobs:
linter-and-tests-on-latest-node:
Expand All @@ -20,11 +20,12 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: latest
node-version: 22.x
- run: node -v
- run: npm ci
- run: npm run lint
- run: npm run pretty:fix
# - run: npm run test:adapter:playwright:example
- run: npm run test
- run: npm run test:storage

Expand All @@ -42,4 +43,4 @@ jobs:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm run test
- run: npm run test:storage
- run: npm run test:storage
2 changes: 1 addition & 1 deletion .github/workflows/publish-newman-reporter-to-npm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ jobs:
- run: npm publish
working-directory: ./packages/newman-reporter-testomatio
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
4 changes: 3 additions & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
lib/template/testomatio.hbs
# fails on CI
lib/pipe/testomatio.js
lib/pipe/testomatio.js

lib/
2 changes: 1 addition & 1 deletion .prettierrc.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module.exports = {
export default {
printWidth: 120,
tabWidth: 2,
useTabs: false,
Expand Down
2 changes: 1 addition & 1 deletion Changelog.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
<!-- pending release updates -->

# next

- move the reporter to ESM (except separate libs from `./packages`)
- added unit tests for Vitest and Playwright
- drop support for Cucumber versions < 7
- add typings
- fix all linter issues and enable linter on CI
- changed path for Mocha and Jamine import, now its `./lib/adapter/mocha/mocha.cjs` and `./lib/adapter/jasmine/jasmine.cjs`


# 1.1.0

- Rewritten logger and artifacts storage. Test reports contain logs, steps, stack traces.
Expand Down
4 changes: 2 additions & 2 deletions build/scripts/edit-package-json.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ fs.readFile(filePath, 'utf8', (err, data) => {
const updatedData = JSON.stringify(packageJson, null, 2);

// Write the updated content back to the file
fs.writeFile(filePath, updatedData, 'utf8', (err) => {
fs.writeFile(filePath, updatedData, 'utf8', err => {
if (err) {
console.error('Error writing the file:', err);
return;
}
console.log(`Updated 'type' to 'commonjs' in ${filePath}`);
});
});
});
3 changes: 1 addition & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,6 @@ Use `/` separator to create a nested rungroup:
TESTOMATIO={API_KEY} TESTOMATIO_RUNGROUP_TITLE="Builds/${BUILD_ID}" <actual run command>
```


#### `TESTOMATIO_SHARED_RUN`

Report parallel execution to the same run matching it by title. **If the run was created more than 20 minutes ago, a new run will be created instead.** To change the timeout use `TESTOMATIO_SHARED_RUN_TIMEOUT` variable.
Expand Down Expand Up @@ -232,4 +231,4 @@ dotenv.config({ path: '.env' }); // or any other path
It is recommended to read `.env` file as early as possible in your application, preferably on test runner initialization.
E.g. in CodeceptJS you can do it in `codecept.conf.js` file. In Playwright: `playwright.config.js`. Jest: `jest.config.js`. Cypress: `cypress.config.js`. And so on.

It is recommended to add `.env` file to `.gitignore` to avoid committing sensitive data to the repository.
It is recommended to add `.env` file to `.gitignore` to avoid committing sensitive data to the repository.
4 changes: 2 additions & 2 deletions docs/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ test('my test', () => {
browser: 'chrome',
server: 'staging',
});
})
});
```

Or in CommonJS style:
Expand All @@ -129,4 +129,4 @@ Supported frameworks:
- 🟢 Jest
- 🟢 Mocha
- 🔴 Playwright (use native functionality instead)
- 🟢 WDIO (everything, except artifacts)
- 🟢 WDIO (everything, except artifacts)
20 changes: 11 additions & 9 deletions lib/adapter/cypress-plugin/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,17 @@ const testomatioReporter = on => {
return this.message;
};
}
const formattedError = error ? {
message: error.message,
name: error.name,
inspect: error.inspect ||
// eslint-disable-next-line
function () {
return this.message;
},
} : undefined;
const formattedError = error
? {
message: error.message,
name: error.name,
inspect: error.inspect ||
// eslint-disable-next-line
function () {
return this.message;
},
}
: undefined;
const screenshots = Array.isArray(results.screenshots)
? results.screenshots
.filter(screenshot => screenshot?.path && screenshot?.path.includes(title) && screenshot?.takenAt)
Expand Down
26 changes: 18 additions & 8 deletions lib/adapter/playwright.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,21 +57,31 @@ class PlaywrightReporter {
const rid = test.id || test.testId || (0, uuid_1.v4)();
/**
* @type {{
* browser?: string,
* dependencies: string[],
* isMobile?: boolean
* metadata: Record<string, any>,
* name: string,
* }}
*/
* browser?: string,
* dependencies: string[],
* isMobile?: boolean
* metadata: Record<string, any>,
* name: string,
* }}
*/
const project = {
browser: test.parent.project().use.defaultBrowserType,
dependencies: test.parent.project().dependencies,
isMobile: test.parent.project().use.isMobile,
metadata: test.parent.project().metadata,
name: test.parent.project().name,
};
const reportTestPromise = this.client.addTestRun(checkStatus(result.status), {
let status = result.status;
// process test.fail() annotation
if (test.expectedStatus === 'failed') {
// actual status = expected
if (result.status === 'failed')
status = 'passed';
// actual status != expected
if (result.status === 'passed')
status = 'failed';
}
const reportTestPromise = this.client.addTestRun(checkStatus(status), {
rid: `${rid}-${project.name}`,
error,
test_id: (0, utils_js_1.getTestomatIdFromTestTitle)(`${title} ${test.tags?.join(' ')}`),
Expand Down
2 changes: 1 addition & 1 deletion lib/adapter/webdriver.js
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ class WebdriverReporter extends reporter_1.default {
}
onRunnerStart() {
// clear dir with artifacts/logs
//
//
utils_js_1.fileSystem.clearDir(constants_js_1.TESTOMAT_TMP_STORAGE_DIR);
}
onTestStart(test) {
Expand Down
4 changes: 2 additions & 2 deletions lib/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ const filesize_1 = require("filesize");
const debug = (0, debug_1.default)('@testomatio/reporter:client');
// removed __dirname usage, because:
// 1. replaced with ESM syntax (import.meta.url), but it throws an error on tsc compilation;
// 2. got error "__dirname already defined" in compiles js code (cjs dir)
// 2. got error "__dirname already defined" in compiles js code (cjs dir)
let listOfTestFilesToExcludeFromReport = null;
/**
* @typedef {import('../types/types.js').TestData} TestData
Expand All @@ -55,7 +55,7 @@ class Client {
* Create a Testomat client instance
* @returns
*/
// eslint-disable-next-line
// eslint-disable-next-line
constructor(params = {}) {
this.paramsForPipesFactory = params;
this.pipeStore = {};
Expand Down
3 changes: 2 additions & 1 deletion lib/pipe/debug.d.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
export class DebugPipe {
constructor(params: any, store: any);
params: any;
store: any;
isEnabled: boolean;
batch: {
isEnabled: any;
Expand All @@ -9,7 +11,6 @@ export class DebugPipe {
batchIndex: number;
};
logFilePath: string;
store: any;
testomatioEnvVars: {};
batchUpload(): Promise<void>;
/**
Expand Down
5 changes: 3 additions & 2 deletions lib/pipe/debug.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,18 @@ const pretty_ms_1 = __importDefault(require("pretty-ms"));
const debug = (0, debug_1.default)('@testomatio/reporter:pipe:debug');
class DebugPipe {
constructor(params, store) {
this.params = params || {};
this.store = store || {};
this.isEnabled = !!process.env.TESTOMATIO_DEBUG || !!process.env.DEBUG;
if (this.isEnabled) {
this.batch = {
isEnabled: params.isBatchEnabled ?? !process.env.TESTOMATIO_DISABLE_BATCH_UPLOAD ?? true,
isEnabled: this.params.isBatchEnabled ?? !process.env.TESTOMATIO_DISABLE_BATCH_UPLOAD ?? true,
intervalFunction: null,
intervalTime: 5000,
tests: [],
batchIndex: 0,
};
this.logFilePath = path_1.default.join(os_1.default.tmpdir(), `testomatio.debug.${Date.now()}.json`);
this.store = store || {};
debug('Creating debug file:', this.logFilePath);
fs_1.default.writeFileSync(this.logFilePath, '');
console.log(constants_js_1.APP_PREFIX, '🪲. Debug created:');
Expand Down
4 changes: 1 addition & 3 deletions lib/pipe/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,7 @@ async function pipesFactory(params, opts) {
...extraPipes,
];
const pipesEnabled = pipes.filter(p => p.isEnabled);
console.log(constants_js_1.APP_PREFIX, picocolors_1.default.cyan('Pipes:'), picocolors_1.default.cyan(pipesEnabled
.map(p => p.toString())
.join(', ') || 'No pipes enabled'));
console.log(constants_js_1.APP_PREFIX, picocolors_1.default.cyan('Pipes:'), picocolors_1.default.cyan(pipesEnabled.map(p => p.toString()).join(', ') || 'No pipes enabled'));
if (!pipesEnabled.length) {
console.log(constants_js_1.APP_PREFIX, picocolors_1.default.dim('If you want to use Testomatio reporter, pass your token as TESTOMATIO env variable'));
}
Expand Down
19 changes: 10 additions & 9 deletions lib/pipe/testomatio.js
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ class TestomatioPipe {
this.axios = axios_1.default.create({
baseURL: `${this.url.trim()}`,
timeout: constants_js_1.AXIOS_TIMEOUT,
proxy: proxy ? {
host: proxy.hostname,
port: parseInt(proxy.port, 10),
protocol: proxy.protocol,
} : false,
proxy: proxy
? {
host: proxy.hostname,
port: parseInt(proxy.port, 10),
protocol: proxy.protocol,
}
: false,
});
// Pass the axios instance to the retry function
(0, axios_retry_1.default)(this.axios, {
Expand Down Expand Up @@ -224,7 +226,8 @@ class TestomatioPipe {
const cancelReporting = this.requestFailures >= parseInt(process.env.TESTOMATIO_MAX_REQUEST_FAILURES, 10);
if (cancelReporting) {
this.reportingCanceledDueToReqFailures = true;
const errorMessage = `⚠️ ${process.env.TESTOMATIO_MAX_REQUEST_FAILURES} requests were failed, reporting to Testomat aborted.`;
let errorMessage = `⚠️ ${process.env.TESTOMATIO_MAX_REQUEST_FAILURES}`;
errorMessage += ' requests were failed, reporting to Testomat aborted.';
console.warn(`${constants_js_1.APP_PREFIX} ${picocolors_1.default.yellow(errorMessage)}`);
}
return cancelReporting;
Expand All @@ -243,9 +246,7 @@ class TestomatioPipe {
}
const json = json_cycle_1.default.stringify(data);
debug('Adding test', json);
return this.axios
.post(`/api/reporter/${this.runId}/testrun`, json, axiosAddTestrunRequestConfig)
.catch(err => {
return this.axios.post(`/api/reporter/${this.runId}/testrun`, json, axiosAddTestrunRequestConfig).catch(err => {
this.requestFailures++;
this.notReportedTestsCount++;
if (err.response) {
Expand Down
Loading
Loading
0