Hi Simon,
On Sun, Jul 3, 2016 at 8:40 AM, Simon Glass sjg@chromium.org wrote:
Sometimes we want to run a command and check that it fails. Add a function to handle this. It can check the return code and also make sure that the output contains a given error message.
Signed-off-by: Simon Glass sjg@chromium.org
test/py/u_boot_utils.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
diff --git a/test/py/u_boot_utils.py b/test/py/u_boot_utils.py index 5d638d9..c834e7b 100644 --- a/test/py/u_boot_utils.py +++ b/test/py/u_boot_utils.py @@ -185,6 +185,28 @@ def cmd(u_boot_console, cmd_str): """ return run_and_log(u_boot_console, cmd_str.split())
+def run_and_log_expect_exception(u_boot_console, cmd, retcode, msg):
- """Run a command which is expected to fail.
nit, in the restrictive case, s/which/that/
- This runs a command and checks that it fails with the expected return code
- and exception method. If not, an exception is raised.
- Args:
u_boot_console: A console connection to U-Boot.cmd: The command to run, as an array of argv[].retcode: Expected non-zero return code from the command.msg: String which should be contained within the command's output.
nit, same as above
- """
- try:
runner = u_boot_console.log.get_runner(cmd[0], sys.stdout)runner.run(cmd)- except Exception as e:
assert(msg in runner.output)
I understand why this is a partial match, as requiring a complete match would result in changing several test cases every time debug output was altered. Not fun.
That said, it seems dangerous. When performing failure tests there are often several failing side-effects. For example, if you are testing hash-mismatches in vboot several code paths may complain about comparison failures, partial output matching may not be able to isolate the failing logic. You may encounter a false positive if any side-effect emits the same expected output.
Would optional parameters make sense? They could allow a tester to restrict matching to the last line of output, or request a complete string match?
- else:
raise Exception('Expected exception, but not raised')
This is a bit vague, does it make sense to include the expected (failing) retcode too?
- finally:
runner.close()ram_base = None def find_ram_base(u_boot_console): """Find the running U-Boot's RAM location. -- 2.8.0.rc3.226.g39d4020
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Let me know if my comments about output matching are too pedantic. Since there are few existing test cases, the last thing I want to do is over-engineer some helpful testing facilities. :)