Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Coveralls isn't solid enough #4694

Closed
bagder opened this issue Dec 10, 2019 · 1 comment
Closed

Coveralls isn't solid enough #4694

bagder opened this issue Dec 10, 2019 · 1 comment
Assignees
@bagder
Labels
build

Comments

@bagder
Copy link
Member

bagder commented Dec 10, 2019

I did this

I've tried to rely on coveralls to tell us data about test coverage. It keeps reporting ridiculous numbers and very often shows bogus red warnings like this:

Screenshot_2019-12-10 Fixes for --disable-doh by MarcelRaad · Pull Request #4692 · curl curl

(screenshot from #4692 - the coverage was never 81% so this claimed 7% decrease is a complete lie)

I expected the following

Reliable info. As this works now, I think it typically does more harm than good: it generates red builds completely arbitrary and even when green the numbers it shows are then hard to trust.

Action

The test coverage is valuable information but changes very rarely. We could probably change the coveralls to get run in a cronjob somewhere every now and then instead. I intend to switch it off from the CI system due to its "unreliablity".
@bagder bagder added the build label Dec 10, 2019
@bagder bagder self-assigned this Dec 10, 2019
bagder added a commit that referenced this issue Dec 10, 2019
@bagder
travis: remove "coverage", make it "torture"
8f6e283 
The coveralls service and test coverage numbers are just too unreliable.
Removed badge from README.md as well.

Fixes #4694
@bagder bagder mentioned this issue Dec 10, 2019
Closed
@dfandrich
Copy link
Contributor

dfandrich commented Dec 10, 2019 via email

@bagder bagder closed this as completed in 83c0e96 Dec 10, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Mar 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bagder bagder

Labels
build
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

travis: remove "coverage", make it "torture" curl/curl
2 participants
@bagder @dfandrich