Repository navigation
Plugin "stepByStepReport" doesn't synchronize screenshots with steps #4405
Description
What are you trying to achieve?
A screenshot should be done after each step in the scenario - there are 4 steps in the scenario, so the screenshot should display a result after each of those 4 steps.
What do you get instead?
All screenshots are done at the very beginning of the test.
E.g. you can't see there filling of the text in the search field or a result of the search query:
record.zip
Provide console output if related. Use
--verbosemode for more details.
$ codeceptjs run --verbose
***************************************
nodeInfo: 18.20.3
osInfo: Linux 6.5 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
cpuInfo: (16) x64 AMD Ryzen 9 7940HS w/ Radeon 780M Graphics
chromeInfo: 126.0.6478.114
edgeInfo: "N/A"
firefoxInfo: undefined
safariInfo: N/A
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v3.6.4 #StandWithUkraine
Using test root "/home/mirao/workspace/tmp"
Helpers: Playwright
Plugins: screenshotOnFail, stepByStepReport, tryTo, retryFailedStep, retryTo, eachElement
StepByStep --
[1] Starting recording promises
Timeouts:
› [Session] Starting singleton browser session
test something
› Screenshot is saving to /home/mirao/workspace/tmp/output/record_c207c723ddc6ff2089642ffbbf5963a68a859f6ffd2558cea60869b5d0bf7dfa/0000.png
› Screenshot is saving to /home/mirao/workspace/tmp/output/record_c207c723ddc6ff2089642ffbbf5963a68a859f6ffd2558cea60869b5d0bf7dfa/0001.png
› Screenshot is saving to /home/mirao/workspace/tmp/output/record_c207c723ddc6ff2089642ffbbf5963a68a859f6ffd2558cea60869b5d0bf7dfa/0002.png
› Screenshot is saving to /home/mirao/workspace/tmp/output/record_c207c723ddc6ff2089642ffbbf5963a68a859f6ffd2558cea60869b5d0bf7dfa/0003.png
I am on page "https://codecept.io"
I fill field "#algolia-search-input", "Step by Step"
I press key "Enter"
I see element "#stepbystepreport"
✔ OK in 1688ms
OK | 1 passed // 2s
◉ Step-by-step preview: file:///home/mirao/workspace/tmp/output/records.htmlProvide test source code if related
Feature('StepByStep');
Scenario('test something', ({ I }) => {
I.amOnPage("https://codecept.io")
I.fillField("#algolia-search-input", "Step by Step")
I.pressKey("Enter");
I.seeElement("#stepbystepreport");
});Details
- CodeceptJS version: 3.6.4
- NodeJS Version: 18.20.3
- Operating System: Ubuntu 22.04
- Playwright 1.44.1
- Configuration file:
import { setHeadlessWhen, setCommonPlugins } from '@codeceptjs/configure';
// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);
// enable all common plugins https://github.com/codeceptjs/configure#setcommonplugins
setCommonPlugins();
export const config: CodeceptJS.MainConfig = {
tests: './*_test.ts',
output: './output',
helpers: {
Playwright: {
browser: 'chromium',
url: 'http://localhost',
show: true
}
},
plugins: {
stepByStepReport: {
enabled: true,
deleteSuccessful: false,
},
},
include: {
I: './steps_file'
},
name: 'tmp'
}
Activity
[-]Plugin "stepByStep" doesn't synchronize screenshots with steps[/-][+]Plugin "stepByStepReport" doesn't synchronize screenshots with steps[/+]mirao commentedon Jun 24, 2024
It seems that CodeceptJS creates a screenshot when adding the scenario steps into a "recorder" and not when executing the steps.
mirao commentedon Jun 24, 2024
Looks like broken by #4385 .
It works for me with 3.6.2 and doesn't work with 3.6.3 and 3.6.4
3.6.2:
CC: @kobenguyent
kobenguyent commentedon Jun 26, 2024
hey @mirao would you mind trying this test version
codeceptjs@3.6.5-beta.1? Hope this resolves the issue.mirao commentedon Jun 26, 2024
@kobenguyent It doesn't work for me with
codeceptjs@3.6.5-beta.1And I'm getting only
record_hash/index.htmlwithout screenshots and withoutrecords.html.This is with 3.6.4:
kobenguyent commentedon Jun 27, 2024
@mirao sorry, wrong push, please try with this test version
codeceptjs@3.6.5-beta.2mirao commentedon Jun 27, 2024
@kobenguyent It works.
But I noticed that if you have
BeforeSuite()in a test, it fails 🐛 (it didn't fail with 3.6.4)Test
output
kobenguyent commentedon Jun 27, 2024
@mirao one more try with
codeceptjs@3.6.5-beta.3, please.mirao commentedon Jun 27, 2024
It works well with
codeceptjs@3.6.5-beta.3. Only issue is thatAfterSuite()isn't omitted from the plugin. It doesn't cause a failure on exit, but it looks nasty. The issue is even in 3.6.4Could you fix it, please?
test:
output:
kobenguyent commentedon Jun 28, 2024
@mirao let's try with codeceptjs@3.6.5-beta.4.
mirao commentedon Jun 28, 2024
@kobenguyent Works well now, thanks.
One more issue: When you have two scenarios and the 1st one fails, then the plugin stops creating screenshots for the 2nd scenario and finally
records.htmlisn't created 🐛Could you look at it? Or may I create a separate issue for it? The bug is even in 3.6.4
test:
output:
mirao commentedon Jun 28, 2024
Well, it seems to be by design.
disableScreenshotOnFailistrueby default. If I set it tofalsethen it works as expected.https://codecept.io/plugins/#stepbystepreport
mirao commentedon Jun 28, 2024
@kobenguyent So you can close the task, everything seems to be solved. Thanks.
kobenguyent commentedon Jun 28, 2024
@mirao thanks for your testing!