github_mirrors/rust-book-cn
1
0
Fork 0
mirror of https://github.com/rust-lang-cn/book-cn.git synced 2025-02-03 07:48:41 +08:00
Code Issues Packages Projects Releases Wiki Activity

initial import/re-write of testing chapter

Browse Source
This commit is contained in:
Steve Klabnik 2016-10-07 15:16:04 -04:00 committed by Carol (Nichols || Goulding)
parent 0097f8c6dc
commit a85af065be
4 changed files with 872 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

279
src/chXX-00-testing.md
View File

@ -1 +1,280 @@
# Testing
> Program testing can be a very effective way to show the presence of bugs, but
> it is hopelessly inadequate for showing their absence.
>
> Edsger W. Dijkstra, "The Humble Programmer" (1972)
Rust is a programming language that cares a lot about correctness. But
correctness is a complex topic, and isn't exactly easy to get right. Rust
places a lot of weight on its type system to help ensure that our programs do
what we intend, but it cannot help with everything. As such, Rust also includes
support for writing software tests in the language itself.
Testing is a skill, and we cannot hope to learn everything about how to write
good tests in one chapter of a book. What we can learn, however, are the
mechanics of Rust's testing facilities. That's what we'll focus on in this
chapter.
## The `test` attribute
At its simplest, a test in Rust is a function that's annotated with the `test`
attribute. Let's make a new project with Cargo called `adder`:
```bash
$ cargo new adder
Created library `adder` project
$ cd adder
```
Cargo will automatically generate a simple test when you make a new project.
Here's the contents of `src/lib.rs`:
```rust
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
}
}
```
For now, let's remove the `mod` bit, and focus on just the function:
```rust
#[test]
fn it_works() {
}
```
Note the `#[test]`. This attribute indicates that this is a test function. It
currently has no body. That's good enough to pass! We can run the tests with
`cargo test`:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///projects/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
Cargo compiled and ran our tests. There are two sets of output here: one
for the test we wrote, and another for documentation tests. We'll talk about
documentation tests later. For now, see this line:
```text
test it_works ... ok
```
Note the `it_works`. This comes from the name of our function:
```rust
fn it_works() {
# }
```
We also get a summary line:
```text
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
```
## The `assert!` macro
So why does our do-nothing test pass? Any test which doesn't `panic!` passes,
and any test that does `panic!` fails. Let's make our test fail:
```rust
#[test]
fn it_works() {
assert!(false);
}
```
`assert!` is a macro provided by Rust which takes one argument: if the argument
is `true`, nothing happens. If the argument is `false`, it will `panic!`. Let's
run our tests again:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///projects/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test it_works ... FAILED
failures:
---- it_works stdout ----
thread 'it_works' panicked at 'assertion failed: false', src/lib.rs:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failures:
it_works
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
error: test failed
```
Rust indicates that our test failed:
```text
test it_works ... FAILED
```
And that's reflected in the summary line:
```text
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
```
## Inverting failure with `should_panic`
We can invert our test's failure with another attribute: `should_panic`:
```rust
#[test]
#[should_panic]
fn it_works() {
assert!(false);
}
```
This test will now succeed if we `panic!` and fail if we complete.
`should_panic` tests can be fragile, as it's hard to guarantee that the test
didn't fail for an unexpected reason. To help with this, an optional `expected`
parameter can be added to the `should_panic` attribute. The test harness will
make sure that the failure message contains the provided text. A safer version
of the example above would be:
```rust
#[test]
#[should_panic(expected = "assertion failed")]
fn it_works() {
assert!(false);
}
```
## Testing equality
Rust provides a pair of macros, `assert_eq!` and `assert_ne!`, that compares
two arguments for equality:
```rust
#[test]
fn it_works() {
assert_eq!("Hello", "Hello");
assert_ne!("Hello", "world");
}
```
These macros expand to something like this:
```rust,ignore
// assert_eq
if left_val == right_val {
panic!("message goes here")
}
// assert_ne
if left_val =! right_val {
panic!("message goes here")
}
```
But they're a bit more convenient than writing this out by hand. These macros
are often used to call some function with some known arguments and compare it
to the expected output, like this:
```rust
pub fn add_two(a: i32) -> i32 {
a + 2
}
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
```
## The `ignore` attribute
Sometimes a few specific tests can be very time-consuming to execute. These
can be disabled by default by using the `ignore` attribute:
```rust
pub fn add_two(a: i32) -> i32 {
a + 2
}
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
#[test]
#[ignore]
fn expensive_test() {
// code that takes an hour to run
}
```
Now we run our tests and see that `it_works` is run, but `expensive_test` is
not:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///projects/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 2 tests
test expensive_test ... ignored
test it_works ... ok
test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
The expensive tests can be run explicitly using `cargo test -- --ignored`:
```bash
$ cargo test -- --ignored
Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test expensive_test ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
The `--ignored` argument is an argument to the test binary, and not to Cargo,
which is why the command is `cargo test -- --ignored`.

130
src/chXX-01-testing.md
View File

@ -1 +1,131 @@
# Unit testing
As we mentioned before, testing is a large discipline, and so different people
can sometimes use different terminology. For our purposes, we tend to place
tests into two main categories: *unit tests* and *integration tests*. Unit
tests tend to be smaller, and more focused. In Rust, they can also test
non-public interfaces. Let's talk more about how to do unit testing in Rust.
## The tests module and `cfg(test)`
Remember when we generated our new project in the last section? Cargo had
generated some stuff for us:
```rust
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
}
}
```
We deleted the module stuff so we could learn more about the mechanics of
tests. But there's a reason that Cargo generated this module for us: it's the
idiomatic way to organize unit tests in Rust. That is, unit tests are:
* Stored inside of the same tree as your source code.
* Placed inside their own module.
For a more realistic example of how this works, consider our `add_two` function
from before:
```rust
pub fn add_two(a: i32) -> i32 {
a + 2
}
#[cfg(test)]
mod tests {
use add_two;
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
}
```
First of all, there's a new attribute, `cfg`. The `cfg` attribute lets us
declare that something should only be included given a certain configuration.
Rust provides the `test` configuration when compiling and running tests. By
using this attribute, Cargo only compiles our test code if we're currently
trying to run the tests. Given that they're not compiled at all during a
regular `cargo build`, this can save compile time. It also ensures that our
tests are entirely left out of the binary, saving space in a non-testing
context.
You'll notice one more change: the `use` declaration. The `tests` module is
only a convention, it's nothing that Rust understands directly. As such, we
have to follow the usual visibility rules. Because we're in an inner module,
we need to bring our test function into scope. This can be annoying if you have
a large module, and so this is a common use of globs. Let's change our
`src/lib.rs` to make use of it:
```rust,ignore
pub fn add_two(a: i32) -> i32 {
a + 2
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
}
```
Note the different `use` line. Now we run our tests:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///projects/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.27 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
It works!
## Testing internal functions
There's controversy within the testing community about unit testing private
functions. Regardless of which testing ideology you adhere to, Rust does allow
you to test them, due to the way that the privacy rules work. Consider this:
```rust
pub fn add_two(a: i32) -> i32 {
internal_adder(a, 2)
}
fn internal_adder(a: i32, b: i32) -> i32 {
a + b
}
#[cfg(test)]
mod tests {
use internal_adder;
#[test]
fn internal() {
assert_eq!(4, internal_adder(2, 2));
}
}
```
In this scenario, we have a non-`pub` function, `internal_adder`. Because tests
are just Rust code, and the `tests` module is just another module, we can
import and call `internal_adder` in a test just fine.

162
src/chXX-02-integration-testing.md
View File

@ -1 +1,163 @@
# Integration testing
In the last section, we talked about unit tests. But there's still that other
category: integration testing. In Rust, an integration test is a test that is
entirely external to your library. It uses it in the same way any other code
would.
Cargo has support for integration tests through the `tests` directory. If you
make one, and put `.rs` files inside, Cargo will compile each of them as an
individual crate. Let's give it a try! First, make a `tests` directory at the
top level of your project, next to `src`. Then, make a new file,
`tests/integration_test.rs`, and put this inside:
```rust,ignore
extern crate adder;
#[test]
fn it_works() {
assert_eq!(4, adder::add_two(2));
}
```
There's some small changes from our previous tests. We now have an `extern
crate adder` at the top. This is because each test in the `tests` directory is
an entirely separate crate, and so we need to import our library. This is also
why `tests` is a suitable place to write integration-style tests: they use the
library like any other consumer of it would.
Let's run them:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///home/steve/tmp/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/lib-c18e7d3494509e74
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
Now we have three sections: our previous test is also run, as well as our new
one.
That's all there is to the `tests` directory. The `tests` module isn't needed
here, since the whole thing is focused on tests.
## Submodules in integration tests
As your integration tests grow, you may want to make more than one file in the
`tests` directory. As we mentioned before, that works well, given that Cargo
treats every file as its own crate. But there's one small trap that can happen.
Imagine we wanted some common helper functions to be shared across our tests.
So we change our test to have a `common` module:
```rust,ignore
extern crate adder;
mod common;
#[test]
fn it_works() {
common::helper();
assert_eq!(4, adder::add_two(2));
}
```
And then, we create a `tests/common.rs` file to hold our common helpers:
```rust
pub fn helper() {
// no implementation for now
}
```
Let's try running this:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///home/steve/tmp/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test tests::internal ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/debug/common-c3635c69f3aeef92
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
Running target/debug/integration_tests-6d6e12b4680b0368
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```
Wait a minute. Now we have four sections?
```text
Running target/debug/common-c3635c69f3aeef92
```
Because `common.rs` is in our `tests` directory, Cargo is also compiling it as
its own crate. Because `common.rs` is so simple, we didn't get an error, but
with more complex code, this might not work. So what can we do?
The key is, always use the `common/mod.rs` form over the `common.rs` form when
making modules in integration tests. If we move `tests/common.rs` to
`tests/common/mod.rs`, we'll go back to our expected output:
```bash
$ mkdir tests/common
$ mv tests/common.rs tests/common/mod.rs
$ cargo test
Compiling adder v0.1.0 (file:///home/steve/tmp/adder)
Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs
Running target/debug/deps/adder-ce99bcc2479f4607
running 1 test
test tests::internal ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/debug/integration_tests-6d6e12b4680b0368
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
```

301
src/chXX-03-documentation-tests.md
View File

@ -1 +1,302 @@
# Documentation Tests
Nothing is better than documentation with examples. Nothing is worse than
examples that don't actually work, because the code has changed since the
documentation has been written. To this end, Rust supports automatically
running examples in your documentation for library crates. Here's a fleshed-out
`src/lib.rs` with examples:
```rust
//! The `adder` crate provides functions that add numbers to other numbers.
//!
//! # Examples
//!
//! ```
//! assert_eq!(4, adder::add_two(2));
//! ```
/// This function adds two to its argument.
///
/// # Examples
///
/// ```
/// use adder::add_two;
///
/// assert_eq!(4, add_two(2));
/// ```
pub fn add_two(a: i32) -> i32 {
a + 2
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
assert_eq!(4, add_two(2));
}
}
```
Note the module-level documentation with `//!` and the function-level
documentation with `///`. Rust's documentation supports Markdown in comments,
and so triple graves mark code blocks. It is conventional to include the
`# Examples` section, exactly like that, with examples following.
Let's run the tests again:
```bash
$ cargo test
Compiling adder v0.1.0 (file:///projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/lib-c18e7d3494509e74
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 2 tests
test add_two_0 ... ok
test _0 ... ok
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured
```
Now we have all three kinds of tests running! Note the names of the
documentation tests: the `_0` is generated for the module test, and `add_two_0`
for the function test. These will auto increment with names like `add_two_1` as
you add more examples.
## Automatic `main` insertion
Let's discuss our sample example documentation:
```rust
/// ```
/// println!("Hello, world");
/// ```
# fn foo() {}
```
You'll notice that you don't need a `fn main()` or anything here. `rustdoc`
will automatically add a `main()` wrapper around your code, using heuristics to
attempt to put it in the right place. For example:
```rust
/// ```
/// use std::rc::Rc;
///
/// let five = Rc::new(5);
/// ```
# fn foo() {}
```
This will end up testing:
```rust
fn main() {
use std::rc::Rc;
let five = Rc::new(5);
}
```
Here's the full algorithm rustdoc uses to preprocess examples:
1. Any leading `#![foo]` attributes are left intact as crate attributes.
2. Some common `allow` attributes are inserted, including
`unused_variables`, `unused_assignments`, `unused_mut`,
`unused_attributes`, and `dead_code`. Small examples often trigger
these lints.
3. If the example does not contain `extern crate`, then `extern crate
<mycrate>;` is inserted (note the lack of `#[macro_use]`).
4. Finally, if the example does not contain `fn main`, the remainder of the
text is wrapped in `fn main() { your_code }`.
This generated `fn main` can be a problem! If you have `extern crate` or a
`mod` statements in the example code that are referred to by `use` statements,
they will fail to resolve unless you include at least `fn main() {}` to inhibit
step 4. `#[macro_use] extern crate` also does not work except at the crate
root, so when testing macros an explicit `main` is always required. It doesn't
have to clutter up your docs, though — keep reading!
## Hiding extraneous code with `#`
Sometimes this algorithm isn't enough, though. For example, all of these code
samples with `///` we've been talking about? The raw text:
```text
/// Some documentation.
# fn foo() {}
```
looks different than the output:
```rust
/// Some documentation.
# fn foo() {}
```
Yes, that's right: we can add lines that start with `# `, and they will be
hidden from the output, but will be used when compiling our code. We can use
this to our advantage. In this case, documentation comments need to apply to
some kind of function, so if We want to show off a documentation comment, I
need to add a little function definition below it. At the same time, it's only
there to satisfy the compiler, so hiding it makes the example more clear. We
can use this technique to explain longer examples in detail, while still
preserving the testability of our documentation.
For example, imagine that we wanted to document this code:
```rust
let x = 5;
let y = 6;
println!("{}", x + y);
```
We might want the documentation to end up looking like this:
> First, we set `x` to five:
>
> ```rust
> let x = 5;
> # let y = 6;
> # println!("{}", x + y);
> ```
>
> Next, we set `y` to six:
>
> ```rust
> # let x = 5;
> let y = 6;
> # println!("{}", x + y);
> ```
>
> Finally, we print the sum of `x` and `y`:
>
> ```rust
> # let x = 5;
> # let y = 6;
> println!("{}", x + y);
> ```
To keep each code block testable, we want the whole program in each block, but
we don't want the reader to see every line every time. Here's what we put in
our source code:
```text
First, we set `x` to five:
```rust
let x = 5;
# let y = 6;
# println!("{}", x + y);
```
Next, we set `y` to six:
```rust
# let x = 5;
let y = 6;
# println!("{}", x + y);
```
Finally, we print the sum of `x` and `y`:
```rust
# let x = 5;
# let y = 6;
println!("{}", x + y);
```
```
By repeating all parts of the example, we can ensure that our example still
compiles, while only showing the parts that are relevant to that part of our
explanation.
Another case where the use of `#` is handy is when you want to ignore
error handling. Lets say you want the following,
```rust,ignore
/// use std::io;
/// let mut input = String::new();
/// try!(io::stdin().read_line(&mut input));
```
The problem is that `try!` returns a `Result<T, E>` and test functions
don't return anything so this will give a mismatched types error.
```rust,ignore
/// A doc test using try!
///
/// ```
/// use std::io;
/// # fn foo() -> io::Result<()> {
/// let mut input = String::new();
/// try!(io::stdin().read_line(&mut input));
/// # Ok(())
/// # }
/// ```
# fn foo() {}
```
You can get around this by wrapping the code in a function. This catches
and swallows the `Result<T, E>` when running tests on the docs. This
pattern appears regularly in the standard library.
## Adding attributes to control documentation testing.
In the first part of the chapter, we talked about attributes that help with
testing:
```rust
#[test]
#[ignore]
fn it_works() {
}
#[should_panic]
fn it_works() {
assert!(false);
}
```
We can use these annotations in documentation tests as well:
```rust
/// ```rust,ignore
/// fn foo() {
/// ```
fn foo() {}
/// ```rust,should_panic
/// assert!(false);
/// ```
fn bar() {}
```
## The `no_run` attribute
There's one attribute that's specific to documentation tests:
```rust
/// ```rust,no_run
/// loop {
/// println!("Hello, world");
/// }
/// ```
# fn foo() {}
```
The `no_run` attribute will compile your code, but not run it. This is
important for examples such as "Here's how to start up a network service,"
which you would want to make sure compile, but might run in an infinite loop!