Testing of Security Manager checklist

Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed

Choose appropriate workflow from the list - with Security Manager included - plugin build used
Download plugin
Open IntelliJ IDE
Remove previously installed UTBot plugin
Install the downloaded plugin

Go through manual scenarios

Basic security

Project with JDK 8 + manual scenario from PR Add SecurityManager support to block suspicious code #622 #625 + default settings
Fuzzing only
Symbolic execution only
Project with JDK 11 + default settings, bug found: Socket connect should be sandboxed #792
Fuzzing only
Symbolic execution only
Concrete executor is invoked from Symbolic execution

Project with JDK 17 - blocked as UTBot is incompatible with JDK 12+

Permissions

add PropertyPermission in ~.utbot\sandbox.policy - check test for property is not in sandbox, execute
add "permission java.io.FilePermission "<>", "read";" - check test for read not in sandbox, execute
add AllPermission in ~.utbot\sandbox.policy - check NO tests are in sandbox, try to execute (carefully!!!)
corrupt permission - check how UTBot fails - found Too many permissions file rereads - can slow down much #796
empty file
remove ~.utbot\sandbox.policy file - check generated tests are in sandbox

Test rendering

JUnit4
JUnit5
TestNG
JUnit5 with parametrization - JUnit5 params dependency is required in default IDEA's Gradle project #620 reproducing
TestNG with parametrization

Code structures

in private method
in static block of a used class - bug found: Implicit unpermitted operations should be sandboxed #791
in compiler of a used class - bug found: Implicit unpermitted operations should be sandboxed #791
in a separate thread - not yet supported - enhancement: Disable tests with unpermitted operations in Threads #895

@Markoutte

@Markoutte @korifey
Please advise if UTBot needs to detect the restricted operations in Threads?
The following method is not executed - AccessControlException is thrown (in logs) - that's Ok.
But the generated tests are not disabled due to Sandbox

public int read() throws Throwable {
        Thread t = new Thread("fileRead") {
            public void run()  {
                try {
                    new File("a.txt").createNewFile();
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            }
        };
        t.start();
        Thread.sleep(500);
        return 10;
    }

Navigation Menu

Repository navigation

Testing of Security Manager checklist #790

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Testing of Security Manager checklist #790

Description

Activity

alisevych commented on Aug 26, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions