Skip to content

Fuzzing only test generation for Builder pattern produces an empty test class #607

Closed as not planned
Closed as not planned
Fuzzing only test generation for Builder pattern produces an empty test class#607
Assignees
EgorkaKulikovsofurihafe
Labels
comp-fuzzingIssue is related to the fuzzingctg-bugIssue is a bug
Milestone
2022.10 Release preparation

Description

@alisevych
alisevych
opened on Jul 28, 2022

Description

Test generation using Fuzzing only mode - the Simplest - for a method instantiating a class using Builder pattern - produces an empty test class.
Sarif report fails to be generated.
FormattingModelInconsistencyException was thrown - twice (not every run).

To Reproduce

Steps to reproduce the behavior:

  1. Run IntelliJ IDEA Ultimate 2022.1.4 with UTBot Java plugin installed
  2. Create a Gradle project with JDK 11
  3. Create the following class:
public class Person {

    private String lastName;
    private String firstName;

    private Person() {
        // private constructor
    }

    public String getLastName() {
        return lastName;
    }

    public String getFirstName() {
        return firstName;
    }

    public static Builder newBuilder() {
        return new Person().new Builder();
    }

    public class Builder {

        private Builder() {
            // private constructor
        }

        public Builder setLastName(String lastName) {
            Person.this.lastName = lastName;

            return this;
        }

        public Builder setFirstName(String firstName) {
            Person.this.firstName = firstName;

            return this;
        }

        public Person build() {
            return Person.this;
        }
    }

    public Person getSomeone(String name) {
        return Person.newBuilder()
                .setFirstName(name)
                .setLastName("Last")
                .build();
    }
}
  1. Generate a test for getSomeone method with UTBot

Expected behavior

Tests are supposed to be generated.

Actual behavior

Empty PersonTest.java class is generated and Error dialog with "Failed to save tests report" is displayed.

Once there was FormattingModelInconsistencyException thrown

Visual proofs (screenshots, logs, images)

image

image

com.intellij.formatting.FormatterImpl$FormattingModelInconsistencyException: Invalid root block PSI element
	at com.intellij.formatting.FormatterImpl.validateModel(FormatterImpl.java:707)
	at com.intellij.formatting.FormatterImpl.format(FormatterImpl.java:221)
	at com.intellij.psi.impl.source.codeStyle.CodeFormatterFacade.processRange(CodeFormatterFacade.java:117)
	at com.intellij.psi.impl.source.codeStyle.CodeFormatterFacade.processElement(CodeFormatterFacade.java:67)
	at com.intellij.formatting.service.CoreFormattingService.formatElement(CoreFormattingService.java:44)
	at com.intellij.formatting.service.FormattingServiceUtil.formatElement(FormattingServiceUtil.java:67)
	at com.intellij.psi.impl.source.codeStyle.CodeStyleManagerImpl.reformat(CodeStyleManagerImpl.java:82)
	at com.intellij.psi.impl.source.codeStyle.CodeStyleManagerImpl.reformat(CodeStyleManagerImpl.java:65)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.reformat(CodeGenerationController.kt:337)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.access$reformat(CodeGenerationController.kt:83)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1$1$2.run(CodeGenerationController.kt:308)
	at com.intellij.openapi.command.WriteCommandAction.lambda$runWriteCommandAction$4(WriteCommandAction.java:362)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$1(WriteCommandAction.java:150)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:1015)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.lambda$doRunWriteCommandAction$2(WriteCommandAction.java:148)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:219)
	at com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:184)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.doRunWriteCommandAction(WriteCommandAction.java:157)
	at com.intellij.openapi.command.WriteCommandAction$BuilderImpl.run(WriteCommandAction.java:124)
	at com.intellij.openapi.command.WriteCommandAction.runWriteCommandAction(WriteCommandAction.java:362)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1$1.run(CodeGenerationController.kt:307)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$run$$inlined$let$lambda$3.compute(actions.kt:58)
	at com.intellij.openapi.application.impl.ApplicationImpl.lambda$runWriteAction$13(ApplicationImpl.java:1024)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteActionWithClass(ApplicationImpl.java:1003)
	at com.intellij.openapi.application.impl.ApplicationImpl.runWriteAction(ApplicationImpl.java:1024)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.run(CodeGenerationController.kt:584)
	at org.utbot.intellij.plugin.generator.CodeGenerationController.access$run(CodeGenerationController.kt:83)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$generateCodeAndReport$1$1.run(CodeGenerationController.kt:293)
	at org.utbot.intellij.plugin.generator.CodeGenerationController$run$$inlined$let$lambda$4.run(ApplicationUtils.kt:59)
	at com.intellij.openapi.application.TransactionGuardImpl.runWithWritingAllowed(TransactionGuardImpl.java:215)
	at com.intellij.openapi.application.TransactionGuardImpl.access$100(TransactionGuardImpl.java:22)
	at com.intellij.openapi.application.TransactionGuardImpl$1.run(TransactionGuardImpl.java:197)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:873)
	at com.intellij.openapi.application.impl.ApplicationImpl$3.run(ApplicationImpl.java:511)
	at com.intellij.openapi.application.impl.FlushQueue.doRun(FlushQueue.java:69)
	at com.intellij.openapi.application.impl.FlushQueue.runNextEvent(FlushQueue.java:112)
	at com.intellij.openapi.application.impl.FlushQueue.flushNow(FlushQueue.java:42)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:776)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:746)
	at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:898)
	at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:746)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$6(IdeEventQueue.java:439)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:803)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$7(IdeEventQueue.java:438)
	at com.intellij.openapi.application.TransactionGuardImpl.performActivity(TransactionGuardImpl.java:106)
	at com.intellij.ide.IdeEventQueue.performActivity(IdeEventQueue.java:604)
	at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$8(IdeEventQueue.java:436)
	at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:873)
	at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:484)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:207)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:128)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:117)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:105)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:92)

Environment

IntelliJ IDEA 2022.1.4
Gradle project with JDK 11

Additional context

When Symbolic engine is turned on - even minimally - a test is generated.
Not reproducing on Gradle project with JDK 8,
Not reproducing on Maven and IntelliJ projects with JDK 11

Activity

  1. alisevych
    added
    ctg-bugIssue is a bug
    comp-fuzzingIssue is related to the fuzzing
    on Jul 28, 2022
  2. alisevych
    assigned
    Markoutte
    on Jul 28, 2022
  3. korifey
    added this to UTBot Javaon Jul 28, 2022
  4. korifey
    moved this to Todo in UTBot Javaon Jul 28, 2022

  5. alisevych commented on Jul 28, 2022

    @alisevych
    alisevych
    on Jul 28, 2022
    MemberAuthor

    @Markoutte Please pay attention to the FormattingModelInconsistencyException - reproduced again with the same scenario.

  6. Markoutte commented on Jul 28, 2022

    @Markoutte
    Markoutte
    on Jul 28, 2022
    Collaborator

    There's no exception before PR #573. Before it creates empty test with error messages about failed concrete execution. @sofurihafe please, take a look.

  7. Markoutte
    assigned
    sofurihafe
    on Jul 28, 2022
  8. EgorkaKulikov
    self-assigned this
    on Jul 28, 2022
  9. alisevych
    added this to the Release preparation milestone on Sep 7, 2022

  10. Markoutte commented on Sep 22, 2022

    @Markoutte
    Markoutte
    on Sep 22, 2022
    Collaborator

    @alisevych fyi, looks like this issue isn't reproducible anymore. Could you verify this?

  11. Markoutte
    removed their assignment
    on Sep 22, 2022

  12. alisevych commented on Sep 22, 2022

    @alisevych
    alisevych
    on Sep 22, 2022
    MemberAuthor

    @Markoutte Closing as not reproducing: a successful test is generated by Fuzzing

  13. alisevych
    closed this as not plannedon Sep 22, 2022
  14. Repository owner moved this from Todo to Done in UTBot Javaon Sep 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

comp-fuzzingIssue is related to the fuzzingctg-bugIssue is a bug

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions