Repository navigation
Manual testing of Release 2022.10.1 #1125
Description
Initial set-up
Check that the IntelliJ Idea UTBot plugin can be successfully installed
- Build from the unit-test-bot/rc10102022 branch
- Download plugin
- Check downloaded zip-file size < 100 MB
- Open IntelliJ IDEA 2022.2.2
- Remove previously installed UTBot plugin
- Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
- Open the project in the IDE
- Install the downloaded plugin
Go through manual scenarios
Manual scenario #1
- Use default plugin settings
- Open the utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java file
- Generate tests for the class
- Remove results
- Generate and Run test for a method
Manual scenario #2
- Use default plugin settings
- Open the utbot-sample/src/main/java/org/utbot/examples/mock/CommonMocksExample.java file
- Generate tests with different Mocking options combinations
Manual scenario #3
- Create a new Gradle project with JDK 17
- Add a simple java file to test
- Generate a test in the existing test root
Manual scenario #4
- Create a new Maven project with JDK 8
- Add a simple java file to test
- Generate a test with a new test root
Manual scenario #5
- Create a new Idea project with JDK 11
- Add a simple java file to test
- Generate tests for several classes
Regression testing
- IDEA 2022.1.4 (JRE 11)
- IDEA 2022.2.2 (JRE 17)
- Sandbox
- Nested calls
- Recursion
- Cycles
- Mocks
- UtUtils class generation
- Summaries: tags, plain text
- Summaries: regions
- Exceptions: thrown, caught, check regions
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
alisevych commentedon Oct 10, 2022
reproducing #1028
reproducing #286
reproducing #747
alisevych commentedon Oct 10, 2022
new bug with Sandboxing and Fuzzing - many tests with same coverage are generated and no minimized
#1129
alisevych commentedon Oct 10, 2022
new bug with Recursion - empty test class and
java.lang.Exception
at com.jetbrains.rd.util.LoggerKt.defaultLogFormat(Logger.kt:80)
at org.utbot.rd.loggers.UtRdConsoleLogger.log(UtRdConsoleLogger.kt:24)
at com.jetbrains.rd.util.SwitchLogger.log(Logger.kt:47)
at org.utbot.instrumentation.instrumentation.et.RuntimeTraceStorage.visit(TraceHandler.kt:162)
at org.utbot.examples.recursion.Recursion.fib(Recursion.java:21)
at org.utbot.examples.recursion.Recursion.fib(Recursion.java:22)
at org.utbot.examples.recursion.Recursion.fib(Recursion.java:22)
...
at org.utbot.instrumentation.instrumentation.InvokeInstrumentation$invoke$2$result$1.invoke-IoAF18A(InvokeInstrumentation.kt:60)
#1130 : reproduced once with Recursion, not reproducing with Cycles
alisevych commentedon Oct 10, 2022
Reopened #273 - please see the comment with stack trace
alisevych commentedon Oct 11, 2022
reproducing #711 - static field and static method
alisevych commentedon Oct 11, 2022
new bug : sandboxed test is not compiling because commented stack trace contains "illegal unicode escape" in "jkmqh\uhauh"
#1134
alisevych commentedon Oct 11, 2022
reproducing #1135
alisevych commentedon Oct 11, 2022
clarification task for Summaries:
regions with Explicitly thrown exception region #1133
to be discussed and corrected
alisevych commentedon Oct 11, 2022
reproducing #1141
alisevych commentedon Oct 12, 2022
reproducing #1098 - on different code, when no tests are generated (timeout is minimal for example)
alisevych commentedon Oct 12, 2022
proceeding with RC testing of build#430
Check with different settings
failed -> new fix in build#434
reproducing StringExamples generation exceeds timeout in 2-3 times #1141
Check other parts
[-]Manual testing of RC build#417[/-][+]Manual testing of RC build#417 - RC build#434[/+]alisevych commentedon Oct 13, 2022
proceeding with RC testing of build#434
Check with different settings - in IDEA 2022.2.3 (new version 05-Oct-2022)
Check other parts
Environmental checks
[-]Manual testing of RC build#417 - RC build#434[/-][+]Manual testing of RC build#417 - RC build#430 - RC build#434[/+][-]Manual testing of RC build#417 - RC build#430 - RC build#434[/-][+]Manual testing of RC[/+]alisevych commentedon Oct 13, 2022
new bug #1149 - reproducing on latest main:
Generate and Run for several classes: tests are generated correctly, but run is executed only for one test class.
alisevych commentedon Oct 13, 2022
new bug: #1160
IDEA dead lock -trying to save SARIF report - when resources folder is removed before generations
alisevych commentedon Oct 14, 2022
new bug: test generated by Symbolic engine for StringExamples.isNotBlank has long string with 6000+ "\u0000"
#1157
[-]Manual testing of RC[/-][+]Manual testing of RC 2022.10.1[/+][-]Manual testing of RC 2022.10.1[/-][+]Manual testing of Release 2022.10.1[/+]