testing
What is code coverage and how do YOU measure it?
I was asked this question regarding our automating testing code coverage. It seems to be that, outside of automated tools, it is more art than science. Are there any real-world examples of how to use code coverage?
What's the difference between a mock & stub?I've read various articles about mocking vs stubbing in testing, including Martin Fowler's Mocks Aren't Stubs, but still don't understand the difference.
toBe(true) vs toBeTruthy() vs toBeTrue()What is the difference between expect(something).toBe(true), expect(something).toBeTruthy() and expect(something).toBeTrue()?
Note that toBeTrue() is a custom matcher introduced in jasmine-matchers among other useful and handy matchers like toHaveMethod() or toBeArrayOfStrings().
The question is meant to be generic, but, as a real-world example, I'm testing that an element is displayed in protractor. Which matcher should I use in this case?
expect(elm.isDisplayed()).toBe(true);
expect(elm.isDisplayed()).toBeTruthy();
expect(elm.isDisplayed()).toBeTrue();
The Android emulator is a bit sluggish. For some devices, like the Motorola Droid and the Nexus One, the app runs faster in the actual device than the emulator. This is a problem when testing games and visual effects.
How do you make the emulator run as fast as possible? I've been toying with its parameters but haven't found a configuration that shows a noticeable improvement yet.
How to run only one unit test class using GradleI am new to Gradle. I use Gradle 1.10 and Ubuntu 13.
I want to know if there's any command to execute only one unit test class, similar to testOnly in SBT.
I worked on an embedded system this summer written in straight C. It was an existing project that the company I work for had taken over. I have become quite accustomed to writing unit tests in Java using JUnit but was at a loss as to the best way to write unit tests for existing code (which needed refactoring) as well as new code added to the system.
Are there any projects out there that make unit testing plain C code as easy as unit testing Java code with JUnit? Any insight that would apply specifically to embedded development (cross-compiling to arm-linux platform) would be greatly appreciated.
How to `go test` all tests in my project?The go test command covers *_test.go files in only one dir.
I want to go test the whole project, which means the test should cover all *_test.go files in the dir ./ and every children tree dir under the dir ./.
What's the command to do this?
How can I write a test which expects an 'Error' to be thrown in Jasmine?I'm trying to write a test for the Jasmine Test Framework which expects an error. At the moment I'm using a Jasmine Node.js integration from GitHub.
In my Node.js module I have the following code:
throw new Error("Parsing is not possible");
Now I try to write a test which expects this error:
describe('my suite...', function() {
[..]
it('should not parse foo', function() {
[..]
expect(parser.parse(raw)).toThrow(new Error("Parsing is not possible"));
});
});
I tried also Error() and some other variants and just can't figure out how to make it work.
I want to get longitude and latitude in Android emulator for testing.
Can any one guide me how to achieve this?
How do I set the location of the emulator to a test position?
How do I assert my exception message with JUnit Test annotation?I have written a few JUnit tests with @Test annotation. If my test method throws a checked exception and if I want to assert the message along with the exception, is there a way to do so with JUnit @Test annotation? AFAIK, JUnit 4.7 doesn't provide this feature but does any future versions provide it? I know in .NET you can assert the message and the exception class. Looking for similar feature in the Java world.
This is what I want:
@Test (expected = RuntimeException.class, message = "Employee ID is null")
public void shouldThrowRuntimeExceptionWhenEmployeeIDisNull() {}
I'm fairly new to the unit testing world, and I just decided to add test coverage for my existing app this week.
This is a huge task, mostly because of the number of classes to test but also because writing tests is all new to me.
I've already written tests for a bunch of classes, but now I'm wondering if I'm doing it right.
When I'm writing tests for a method, I have the feeling of rewriting a second time what I already wrote in the method itself.
My tests just seems so tightly bound to the method (testing all codepath, expecting some inner methods to be called a number of times, with certain arguments), that it seems that if I ever refactor the method, the tests will fail even if the final behavior of the method did not change.
This is just a feeling, and as said earlier, I have no experience of testing. If some more experienced testers out there could give me advices on how to write great tests for an existing app, that would be greatly appreciated.
Edit : I would love to thank Stack Overflow, I had great inputs in less that 15 minutes that answered more of the hours of online reading I just did.
How do you print in a Go test using the "testing" package?I'm running a test in Go with a statement to print something (i.e. for debugging of tests) but it's not printing anything.
func TestPrintSomething(t *testing.T) {
fmt.Println("Say hi")
}
When I run go test on this file, this is the output:
ok command-line-arguments 0.004s
The only way to really get it to print, as far as I know, is to print it via t.Error(), like so:
func TestPrintSomethingAgain(t *testing.T) {
t.Error("Say hi")
}
Which outputs this:
Say hi
--- FAIL: TestPrintSomethingAgain (0.00 seconds)
foo_test.go:35: Say hi
FAIL
FAIL command-line-arguments 0.003s
gom: exit status 1
I've Googled and looked through the manual but didn't find anything.
How to properly assert that an exception gets raised in pytest?# coding=utf-8
import pytest
def whatever():
return 9/0
def test_whatever():
try:
whatever()
except ZeroDivisionError as exc:
pytest.fail(exc, pytrace=True)
================================ test session starts =================================
platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2
plugins: django, cov
collected 1 items
pytest_test.py F
====================================== FAILURES ======================================
___________________________________ test_whatever ____________________________________
def test_whatever():
try:
whatever()
except ZeroDivisionError as exc:
> pytest.fail(exc, pytrace=True)
E Failed: integer division or modulo by zero
pytest_test.py:12: Failed
============================== 1 failed in 1.16 seconds ==============================
How to make pytest print traceback, so I would see where in the whatever function an exception was raised?
I have read this post about how to test private methods. I usually do not test them, because I always thought it's faster to test only public methods that will be called from outside the object. Do you test private methods? Should I always test them?
How to unit test abstract classes: extend with stubs?I was wondering how to unit test abstract classes, and classes that extend abstract classes.
Should I test the abstract class by extending it, stubbing out the abstract methods, and then test all the concrete methods? Then only test the methods I override, and test the abstract methods in the unit tests for objects that extend my abstract class?
Should I have an abstract test case that can be used to test the methods of the abstract class, and extend this class in my test case for objects that extend the abstract class?
Note that my abstract class has some concrete methods.
Writing unit tests in Python: How do I start?I completed my first proper project in Python and now my task is to write tests for it.
Since this is the first time I did a project, this is the first time I would be writing tests for it.
The question is, how do I start? I have absolutely no idea. Can anyone point me to some documentation/ tutorial/ link/ book that I can use to start with writing tests (and I guess unit testing in particular)
Any advice will be welcomed on this topic.
How do I run all Python unit tests in a directory?I have a directory that contains my Python unit tests. Each unit test module is of the form test_*.py. I am attempting to make a file called all_test.py that will, you guessed it, run all files in the aforementioned test form and return the result. I have tried two methods so far; both have failed. I will show the two methods, and I hope someone out there knows how to actually do this correctly.
For my first valiant attempt, I thought "If I just import all my testing modules in the file, and then call this unittest.main() doodad, it will work, right?" Well, turns out I was wrong.
import glob
import unittest
testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
if __name__ == "__main__":
unittest.main()
This did not work, the result I got was:
$ python all_test.py
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
For my second try, I though, ok, maybe I will try to do this whole testing thing in a more "manual" fashion. So I attempted to do that below:
import glob
import unittest
testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
[__import__(str) for str in module_strings]
suites = [unittest.TestLoader().loadTestsFromName(str) for str in module_strings]
[testSuite.addTest(suite) for suite in suites]
print testSuite
result = unittest.TestResult()
testSuite.run(result)
print result
#Ok, at this point I have a result
#How do I display it as the normal unit test command line output?
if __name__ == "__main__":
unittest.main()
This also did not work, but it seems so close!
$ python all_test.py
<unittest.TestSuite tests=[<unittest.TestSuite tests=[<unittest.TestSuite tests=[<test_main.TestMain testMethod=test_respondes_to_get>]>]>]>
<unittest.TestResult run=1 errors=0 failures=0>
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
I seem to have a suite of some sort, and I can execute the result. I am a little concerned about the fact that it says I have only run=1, seems like that should be run=2, but it is progress. But how do I pass and display the result to main? Or how do I basically get it working so I can just run this file, and in doing so, run all the unit tests in this directory?
What is the most exact way of seeing how long something, for example a method call, took in code?
The easiest and quickest I would guess is this:
DateTime start = DateTime.Now;
{
// Do some work
}
TimeSpan timeItTook = DateTime.Now - start;
But how exact is this? Are there better ways?
Test process.env with JestI have an application that depends on environmental variables like:
const APP_PORT = process.env.APP_PORT || 8080;
And I would like to test that for example:
process.env.APP_PORTHow can I achieve this with Jest? Can I set these process.env variables before each test or should I mock it somehow maybe?
When I run tests I get the error "Command line is too long". It works if I set the "Shorten command line" method in the Run/Debug configuration to "JAR manifest" for the specific method or class, but how do I set it for the whole project or is there an IDE global setting for it?
How do you run a single test/spec file in RSpec?I want to be able to run a single spec file's tests — for the one file I'm editing, for example. rake spec executes all the specs. My project is not a Rails project, so rake spec:doc doesn't work.
Don't know if this matters, but here is my directory structure.
./Rakefile ./lib ./lib/cushion.rb ./lib/cushion ./lib/cushion/doc.rb ./lib/cushion/db.rb ./spec ./spec/spec.opts ./spec/spec_helper.rb ./spec/db_spec.rbWhat is the difference between unit tests and functional tests?
What is the difference between unit tests and functional tests? Can a unit test also test a function?
Gradle: How to Display Test Results in the Console in Real Time?I would like to see test results ( system.out/err, log messages from components being tested ) as they run in the same console I run:
gradle test
And not wait until tests are done to look at the test reports ( that are only generated when tests are completed, so I can't "tail -f" anything while tests are running )
What are the differences between unit tests, integration tests, smoke tests, and regression tests?What are unit tests, integration tests, smoke tests, and regression tests? What are the differences between them and which tools can I use for each of them?
For example, I use JUnit and NUnit for unit testing and integration testing. Are there any tools for the last two, smoke testing or regression testing?
Rspec: "array.should == another_array" but without concern for orderI often want to compare arrays and make sure that they contain the same elements, in any order. Is there a concise way to do this in RSpec?
Here are methods that aren't acceptable:
#to_setFor example:
expect(array.to_set).to eq another_array.to_set
or
array.to_set.should == another_array.to_set
This fails when the arrays contain duplicate items.
#sortFor example:
expect(array.sort).to eq another_array.sort
or
array.sort.should == another_array.sort
This fails when the arrays elements don't implement #<=>
I'm trying to make UI test for my android app in Kotlin. Since the new system using ActivityTestRule, I can't make it work: it compiles correctly, and at runtime, I get:
java.lang.Exception: The @Rule 'mActivityRule' must be public.
at org.junit.internal.runners.rules.RuleFieldValidator.addError(RuleFieldValidator.java:90)
at org.junit.internal.runners.rules.RuleFieldValidator.validatePublic(RuleFieldValidator.java:67)
at org.junit.internal.runners.rules.RuleFieldValidator.validateField(RuleFieldValidator.java:55)
at org.junit.internal.runners.rules.RuleFieldValidator.validate(RuleFieldValidator.java:50)
at org.junit.runners.BlockJUnit4ClassRunner.validateFields(BlockJUnit4ClassRunner.java:170)
at org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.ParentRunner.validate(ParentRunner.java:344)
at org.junit.runners.ParentRunner.<init>(ParentRunner.java:74)
at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:55)
at android.support.test.internal.runner.junit4.AndroidJUnit4ClassRunner.<init>(AndroidJUnit4ClassRunner.java:38)
at android.support.test.runner.AndroidJUnit4.<init>(AndroidJUnit4.java:36)
at java.lang.reflect.Constructor.constructNative(Native Method)
at java.lang.reflect.Constructor.newInstance(Constructor.java:417)
at android.support.test.internal.runner.junit4.AndroidAnnotatedBuilder.buildAndroidRunner(AndroidAnnotatedBuilder.java:57)
at android.support.test.internal.runner.junit4.AndroidAnnotatedBuilder.runnerForClass(AndroidAnnotatedBuilder.java:45)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:29)
at org.junit.runner.Computer.getRunner(Computer.java:38)
at org.junit.runner.Computer$1.runnerForClass(Computer.java:29)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:98)
at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:84)
at org.junit.runners.Suite.<init>(Suite.java:79)
at org.junit.runner.Computer.getSuite(Computer.java:26)
at android.support.test.internal.runner.TestRequestBuilder.classes(TestRequestBuilder.java:691)
at android.support.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:654)
at android.support.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:329)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:226)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1584)
Here is how I declared mActivityRule:
RunWith(javaClass<AndroidJUnit4>())
LargeTest
public class RadisTest {
Rule
public val mActivityRule: ActivityTestRule<MainActivity> = ActivityTestRule(javaClass<MainActivity>())
...
}
It is already public :/
Stopping an Android app from consoleIs it possible to stop an Android app from the console? Something like:
adb stop com.my.app.package
It would speed up our testing process so much. Right now we uninstall/install the app each time to make sure the manual test cases start with a clean state.
What's the difference between unit, functional, acceptance, and integration tests?What is the difference between unit, functional, acceptance, and integration testing (and any other types of tests that I failed to mention)?
How to check if a string array contains one string in JavaScript?I have a string array and one string. I'd like to test this string against the array values and apply a condition the result - if the array contains the string do "A", else do "B".
How can I do that?
Testing HTML email renderingAre there any good tools to easily test how HTML email will look across different email clients? I prefer something with instant feed back rather than a submit and wait service like http://litmusapp.com Or at the very least a way to test the Outlook 2007/MS Word rendering?
I found this related question but it doesn't specifically address testing. What guidelines for HTML email design are there?
testing