Option for Istanbul coverage to include all source files (as opposed to only files loaded during test)

Please provide us with the following information:

OS? Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)
Any
Versions. Please run ng --version. If there's nothing outputted, please run
in a Terminal: node --version and paste the result here:

angular-cli: 1.0.0-beta.11-webpack.2
node: 6.2.2
os: linux x64

Repro steps. Was this an app that wasn't created using the CLI? What change did you
do on your code? etc.

Create an application. Test some, but not all of the source files. Generated coverage will not include files that were not loaded by the tests.

The log given by the failure. Normally this include a stack trace and some
more information.

N/A

Mention any other details that might be useful.

Thanks! We'll be in touch soon.

Similar to gotwarlost/istanbul#275 and karma-runner/karma-coverage#125

+1
karma-coverage has an option includeAllSources which could be used.

If anyone has is willing to do a PR, or point me in the right direction to do it with webpack/karma-remap-istanbul, I'd be happy to look at it.

@filipesilva

@filipesilva what exactly are you asking for I am confused.

Given that I understand webpack a little better now the includeAllSources option in karma would not work. Under the hood that is including all files that were instrumented. Webpack won't instrument code that isn't loaded so includeAllSources would not see it.

Instead I ended up using a simpler workaround. I added a single file:

src/app.module.spec.ts

which is simply (e.g. no test suites or cases):

import './app.module';

This file will cause webpack to load the entire application. The istanbul coverage will only be generated for local code (since node_modules is excluded). As far as I can tell this is a pretty ideal solution with 2 caveats:

It is still possible for code to be missed in coverage if that code is not included in the application at all. One could potentially mitigate this with a webpack plugin that checked for unused code in ./src and reported it.
It will cause an increase in test times depending on the size of the application.

Let me know if you want me to make a pull request with this particular solution (or go ahead and implement it as it is simple enough). I wasn't sure creating a file named app.module.spec.ts would be universally accepted (some might be confused by its presence) and was hoping that someone might have a better, more invisible, idea on how to accomplish the same thing.

@westonpace

@westonpace I see how your solution works. It's something that's worth discussing before being implemented though...

A more direct use of your solution would be to add import './app.module'; to src/test.ts instead.

I think the biggest drawback is now that the whole app is indeed loaded on every test, whereas we only realistically want it for code coverage tests.

Not sure of what the best approach is for now, but at least we have a workaround. Let's let this issue sit open for a while, see what people say.

Another option may be to modify the following line in test.ts to require everything:

.then(() => require.context('./', true, /\.spec\.ts/))
// Becomes
.then(() => require.context('./app', true, /\.ts/))

If you use lazy loaded modules importing AppModule won't include everything.

It seems like the main downside with showing coverage for everything is that you may have files that can't be covered, or don't make sense to be covered that will be included in the report, skewing your numbers a little.

@seeruk

I'm successfully using

.then(() => require.context('./', true, /\/app\/.*\.ts$/))

in src/test.ts.
@seeruk's solution seems to trigger a weback bug in the context loader where files are loaded as directories. Putting the 'app' path part in the regex works.

Aah, perhaps it needed a slash on the end of the path. I'd not actually tried that solution with angular-cli, I'm just doing something similar in the boilerplate I've been making.

I am not using angular-cli. Is there an option to include all source files in coverage while using karma-test-shim.js to load systemjs configuration and all test files.

@filipesilva

@filipesilva

I want to suggest separated file coverage.ts near test.ts and coverage option in .angular-cli.json.
In coverage.ts we can do all stuff related to full code coverage and developers can configure what file use for coverage via coverage option (use "coverage": "coverage.ts" - for full coverage, and "coverage": "test.ts" keeps current behaviour).

@arusakov

@arusakov I have implemented something similar to what you are suggesting in my own application. The downfall to this is that I then have two test start points that need to stay in sync when things might be changed.
I implemented a coverage.ts in the same folder as test.ts and updated the karma-conf.js to actually change the files array and preprocessors object to reflect the new start point if config.angularCli.codeCoverage is true.
It works pretty well, but having a flag available to test.ts to allow for switching the context based on that flag I think would be slightly more ideal than having two separate files.

Since this is still open, I intend to solve this issue in my project by creating a schematic that adds tests where theyre missing, I found that even if the spec files are empty as long as they import the file it seems to generate correct coverage.

I've had good luck using the karma-sabarivka-reporter plugin to pick up missed source files into coverage.

test(web-client): fix coverage reporting

test(web-client): fix coverage reporting (#37)

Has there been any progress or changes in this issue over the past few Angular versions?

Seems like the two primary options to force coverage reporting on all sources, regardless of if they have a .spec file, are:

Add an app.module.spec.ts file that just imports app.module.ts.
Use https://github.com/kopach/karma-sabarivka-reporter

Is that still the situation today?

Navigation Menu

Repository navigation

Option for Istanbul coverage to include all source files (as opposed to only files loaded during test) #1735

Please provide us with the following information:

18 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Option for Istanbul coverage to include all source files (as opposed to only files loaded during test) #1735

Description

Please provide us with the following information:

Activity

jtesser commented on Oct 20, 2016

filipesilva commented on Oct 24, 2016

jtesser commented on Oct 26, 2016

westonpace commented on Oct 27, 2016

westonpace commented on Oct 27, 2016

filipesilva commented on Nov 2, 2016

seeruk commented on Nov 4, 2016

blaugold commented on Nov 7, 2016

seeruk commented on Nov 7, 2016

rashmi-dey commented on Mar 2, 2017

arusakov commented on Mar 25, 2017

kpaxton commented on May 16, 2017

18 remaining items

CraigComeOnThisNameCantBeTaken commented on Oct 27, 2019

johnthagen commented on Feb 23, 2020

chucknelson commented on Aug 16, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions