Repository navigation
Improve sandbox-relative description in generated tests #782
Description
Description
After #625 was implemented methods that try to run some unsafe code now generates into tests like this:
@Disabled("Disabled due to sandbox")
public void testPropertyWithBlankString() {
SecurityCheck securityCheck = new SecurityCheck();
/* This test fails because method [com.company.security.SecurityCheck.property] produces [java.security.AccessControlException: access denied ("java.util.PropertyPermission" " " "read")]
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
java.security.AccessController.checkPermission(AccessController.java:886)
java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1294)
java.lang.System.getProperty(System.java:719)
com.company.security.SecurityCheck.property(SecurityCheck.java:32) */
}We have several problems with this view:
- User have no information how to change settings of security manager
- TestNG has no
@Disabledannotation - Is it good idea to print stacktrace?
Expected behavior
Generated test should be improved. We can add little doc or even hyperlink to add permissions into correct file.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
alisevych commentedon Aug 24, 2022
@Markoutte @EgorkaKulikov
Could you please also consider the following:
Disabling tests is made in a bit different way in different test frameworks:
JUnit4 has only
@Ignore("<comment>")JUnit5 has
@Disabled("<comment>")TestNG has possibility to disable tests inside
@Test( enabled=false ), there can be also description parameter inside, that Summaries is using. So need to render it once and correctly.How these tests will be rendered when parameterization is on?
alisevych commentedon Aug 25, 2022
@Markoutte @EgorkaKulikov
Tested cases from previous comment - tests are rendered correctly.
alisevych commentedon Sep 26, 2022
"This test fails.." is not a correct note. The test would pass if enabled.
"The test was not generated..." would be more clear. Or "The test generation stopped on unpermitted operation..."
Also it would be great to suggest the user to allow UTBot to generate and run the tests for these unsafe operations - @Markoutte 's idea.