[U-Boot] [PATCH 0/6] test.py fixes & enhancements
This builds on my previous patch "Implement pytest-based test infrastructure". It converts a few more tests so provides a few more examples.
Stephen Warren (6): test/py: require .config to exist test/py: parse include/autoconf.mk for config options test/py: limit outstanding TX data test/py: print implicit shell prompt later test/py: replace cmd_repeat.sh with Python test/py: convert most of test/command_ut.c to Python
test/cmd_repeat.sh | 29 -------- test/command_ut.c | 133 ---------------------------------- test/py/conftest.py | 18 +++-- test/py/test_env.py | 26 ++++++- test/py/test_hush_if_test.py | 137 +++++++++++++++++++++++++++++++++++ test/py/test_md.py | 10 +++ test/py/test_shell_basics.py | 31 ++++++++ test/py/uboot_console_base.py | 31 ++++---- test/py/uboot_console_exec_attach.py | 6 +- test/py/uboot_console_sandbox.py | 2 +- 10 files changed, 233 insertions(+), 190 deletions(-) delete mode 100755 test/cmd_repeat.sh create mode 100644 test/py/test_hush_if_test.py create mode 100644 test/py/test_shell_basics.py
From: Stephen Warren swarren@nvidia.com
If .config doesn't exist, the test will currently throw an exception because the shell prompt value can't be looked up. It's not obvious how to resolve this. Fix the code to require that the .config file exist in the build directory rather than ignoring the issue when it's missing. Make the error message reference the fact that U-Boot doesn't appear to be built, and mention how to solve that.
Signed-off-by: Stephen Warren swarren@nvidia.com --- test/py/conftest.py | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/test/py/conftest.py b/test/py/conftest.py index 4b40bdd89a60..da8618ba998c 100644 --- a/test/py/conftest.py +++ b/test/py/conftest.py @@ -99,15 +99,16 @@ def pytest_configure(config): ubconfig.__dict__[k] = v
dot_config = build_dir + "/.config" - if os.path.exists(dot_config): - with open(dot_config, "rt") as f: - ini_str = "[root]\n" + f.read() - ini_sio = StringIO.StringIO(ini_str) - parser = ConfigParser.RawConfigParser() - parser.readfp(ini_sio) - ubconfig.buildconfig = dict(parser.items("root")) - else: - ubconfig.buildconfig = dict() + if not os.path.exists(dot_config): + raise Exception(".config file does not exist; " + + "try passing --build option?") + + with open(dot_config, "rt") as f: + ini_str = "[root]\n" + f.read() + ini_sio = StringIO.StringIO(ini_str) + parser = ConfigParser.RawConfigParser() + parser.readfp(ini_sio) + ubconfig.buildconfig = dict(parser.items("root"))
ubconfig.test_py_dir = test_py_dir ubconfig.source_dir = source_dir
On Tue, Nov 17, 2015 at 08:19:59PM -0700, Stephen Warren wrote:
From: Stephen Warren swarren@nvidia.com
If .config doesn't exist, the test will currently throw an exception because the shell prompt value can't be looked up. It's not obvious how to resolve this. Fix the code to require that the .config file exist in the build directory rather than ignoring the issue when it's missing. Make the error message reference the fact that U-Boot doesn't appear to be built, and mention how to solve that.
Signed-off-by: Stephen Warren swarren@nvidia.com
Reviewed-by: Tom Rini trini@konsulko.com
From: Stephen Warren swarren@nvidia.com
Many config options aren't yet converted to Kconfig. Update the test scripts to parse include/autoconf.mk to pick up the non-converted options. This will allow tests to be marked as depending on those options.
Signed-off-by: Stephen Warren swarren@nvidia.com --- test/py/conftest.py | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/test/py/conftest.py b/test/py/conftest.py index da8618ba998c..27f6f4d69c15 100644 --- a/test/py/conftest.py +++ b/test/py/conftest.py @@ -98,17 +98,20 @@ def pytest_configure(config): continue ubconfig.__dict__[k] = v
- dot_config = build_dir + "/.config" - if not os.path.exists(dot_config): - raise Exception(".config file does not exist; " + - "try passing --build option?") - - with open(dot_config, "rt") as f: - ini_str = "[root]\n" + f.read() - ini_sio = StringIO.StringIO(ini_str) - parser = ConfigParser.RawConfigParser() - parser.readfp(ini_sio) - ubconfig.buildconfig = dict(parser.items("root")) + ubconfig.buildconfig = dict() + + for conf_file in (".config", "include/autoconf.mk"): + dot_config = build_dir + "/" + conf_file + if not os.path.exists(dot_config): + raise Exception(conf_file + " does not exist; " + + "try passing --build option?") + + with open(dot_config, "rt") as f: + ini_str = "[root]\n" + f.read() + ini_sio = StringIO.StringIO(ini_str) + parser = ConfigParser.RawConfigParser() + parser.readfp(ini_sio) + ubconfig.buildconfig.update(parser.items("root"))
ubconfig.test_py_dir = test_py_dir ubconfig.source_dir = source_dir
On Tue, Nov 17, 2015 at 08:20:00PM -0700, Stephen Warren wrote:
From: Stephen Warren swarren@nvidia.com
Many config options aren't yet converted to Kconfig. Update the test scripts to parse include/autoconf.mk to pick up the non-converted options. This will allow tests to be marked as depending on those options.
Signed-off-by: Stephen Warren swarren@nvidia.com
Reviewed-by: Tom Rini trini@konsulko.com
U-Boot echoes back all characters it receives on the console. At best, with zero overhead, it could keep up with the incoming data stream. However, in practice there's always some delay between receiving a character from the RX side of the console and handing it off to the TX side of the console, which means the TX patch cannot keep up with the RX path. This is particularly true due to U-Boot's unbuffered polling IO model. This is exacerbated when the target performs slow operations for each character, such as drawing on the LCD.
When sending large command-lines, larger than the UART HW's RX FIFO size, this issue can lead to dropped characters, and hence test failures.
To work around this, modify the test scripts to that they limit the number of outstanding characters, i.e. those sent to the target but not yet echo'd back. This avoids
Signed-off-by: Stephen Warren swarren@wwwdotorg.org --- test/py/uboot_console_base.py | 27 ++++++++++++--------------- test/py/uboot_console_exec_attach.py | 6 +++++- test/py/uboot_console_sandbox.py | 2 +- 3 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/test/py/uboot_console_base.py b/test/py/uboot_console_base.py index dfd986860e75..4bf9e8303633 100644 --- a/test/py/uboot_console_base.py +++ b/test/py/uboot_console_base.py @@ -21,9 +21,10 @@ class ConsoleDisableCheck(object): self.console.disable_check_count[self.check_type] -= 1
class ConsoleBase(object): - def __init__(self, log, config): + def __init__(self, log, config, max_fifo_fill): self.log = log self.config = config + self.max_fifo_fill = max_fifo_fill
self.logstream = self.log.get_stream("console", sys.stdout)
@@ -63,22 +64,19 @@ class ConsoleBase(object): bad_patterns.append(pattern_error_notification) bad_pattern_ids.append("Error notification") try: - if cmd: - self.p.send(cmd) - try: - m = self.p.expect([re.escape(cmd)] + bad_patterns) - if m != 0: - self.at_prompt = False - raise Exception("Bad pattern found on console: " + - bad_pattern_ids[m - 1]) - except Exception as ex: + self.at_prompt = False + while cmd: + # Limit max outstanding data, so UART FIFOs don't overflow + chunk = cmd[:self.max_fifo_fill] + cmd = cmd[self.max_fifo_fill:] + self.p.send(chunk) + m = self.p.expect([re.escape(chunk)] + bad_patterns) + if m != 0: self.at_prompt = False - print cmd - self.logstream.write(cmd, implicit=True) - raise + raise Exception("Bad pattern found on console: " + + bad_pattern_ids[m - 1]) self.p.send("\n") if not wait_for_prompt: - self.at_prompt = False return m = self.p.expect([self.prompt_escaped] + bad_patterns) if m != 0: @@ -88,7 +86,6 @@ class ConsoleBase(object): self.at_prompt = True return self.p.before.strip() except Exception as ex: - self.at_prompt = False self.log.error(str(ex)) self.cleanup_spawn() raise diff --git a/test/py/uboot_console_exec_attach.py b/test/py/uboot_console_exec_attach.py index 7960d66107c3..0beaa4fe7102 100644 --- a/test/py/uboot_console_exec_attach.py +++ b/test/py/uboot_console_exec_attach.py @@ -7,7 +7,11 @@ def cmdline(app, args):
class ConsoleExecAttach(ConsoleBase): def __init__(self, log, config): - super(ConsoleExecAttach, self).__init__(log, config) + # The max_fifo_fill value might need tweaking per-board/-SoC? + # 1 would be safe anywhere, but is very slow (a pexpect issue?). + # 16 is a common FIFO size. + # HW flow control would mean this could be infinite. + super(ConsoleExecAttach, self).__init__(log, config, max_fifo_fill=16)
self.log.action("Flashing U-Boot") cmd = ["uboot-test-flash", config.board_type, config.board_identity] diff --git a/test/py/uboot_console_sandbox.py b/test/py/uboot_console_sandbox.py index c3aae3862ca9..ed3ffa5b90b1 100644 --- a/test/py/uboot_console_sandbox.py +++ b/test/py/uboot_console_sandbox.py @@ -4,7 +4,7 @@ from uboot_console_base import ConsoleBase
class ConsoleSandbox(ConsoleBase): def __init__(self, log, config): - super(ConsoleSandbox, self).__init__(log, config) + super(ConsoleSandbox, self).__init__(log, config, max_fifo_fill=1024)
def get_spawn(self): return pexpect.spawn(self.config.build_dir + "/u-boot")
Whenever a test is about to start, the test scripts log an "implicit" version of the shell prompt. This ensures that each test's log section includes the prompt, which makes it easier to read.
However, pytest apparently doesn't set up stdout capturing for skipped tests. If we print this implicit prompt before we've determined whether the test is skipped, then it may/will appear as stdout in ther terminal where the test script was run, which looks messy.
To avoid this, only print this prompt after we've evaluated whether the test is to be skipped. Note that internally, pytest.skip() raises an exception, which causes the moved code not to execute, now that it's later in the execution path.
Signed-off-by: Stephen Warren swarren@wwwdotorg.org --- test/py/conftest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/test/py/conftest.py b/test/py/conftest.py index 27f6f4d69c15..7d11278b5f65 100644 --- a/test/py/conftest.py +++ b/test/py/conftest.py @@ -184,11 +184,11 @@ def setup_envspec(item):
def pytest_runtest_setup(item): log.start_section(item.name) - if console.at_prompt: - console.logstream.write(console.prompt, implicit=True) setup_boardspec(item) setup_buildconfigspec(item) setup_envspec(item) + if console.at_prompt: + console.logstream.write(console.prompt, implicit=True)
def pytest_runtest_protocol(item, nextitem): reports = runtestprotocol(item, nextitem=nextitem)
On Tue, Nov 17, 2015 at 08:20:02PM -0700, Stephen Warren wrote:
Whenever a test is about to start, the test scripts log an "implicit" version of the shell prompt. This ensures that each test's log section includes the prompt, which makes it easier to read.
However, pytest apparently doesn't set up stdout capturing for skipped tests. If we print this implicit prompt before we've determined whether the test is skipped, then it may/will appear as stdout in ther terminal where the test script was run, which looks messy.
To avoid this, only print this prompt after we've evaluated whether the test is to be skipped. Note that internally, pytest.skip() raises an exception, which causes the moved code not to execute, now that it's later in the execution path.
Signed-off-by: Stephen Warren swarren@wwwdotorg.org
Reviewed-by: Tom Rini trini@konsulko.com
From: Stephen Warren swarren@nvidia.com
This moves this test into the new infra-structure, and also allows it to work on real hardware, not just sandbox.
Signed-off-by: Stephen Warren swarren@nvidia.com --- test/cmd_repeat.sh | 29 ----------------------------- test/py/test_md.py | 10 ++++++++++ 2 files changed, 10 insertions(+), 29 deletions(-) delete mode 100755 test/cmd_repeat.sh
diff --git a/test/cmd_repeat.sh b/test/cmd_repeat.sh deleted file mode 100755 index 990e79900f47..000000000000 --- a/test/cmd_repeat.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/sh - -# Test for U-Boot cli including command repeat - -BASE="$(dirname $0)" -. $BASE/common.sh - -run_test() { - ./${OUTPUT_DIR}/u-boot <<END -setenv ctrlc_ignore y -md 0 - -reset -END -} -check_results() { - echo "Check results" - - grep -q 00000100 ${tmp} || fail "Command did not repeat" -} - -echo "Test CLI repeat" -echo -tmp="$(tempfile)" -build_uboot -run_test >${tmp} -check_results ${tmp} -rm ${tmp} -echo "Test passed" diff --git a/test/py/test_md.py b/test/py/test_md.py index 49cdd2685234..9667fd4f2217 100644 --- a/test/py/test_md.py +++ b/test/py/test_md.py @@ -10,3 +10,13 @@ def test_md(uboot_console): uboot_console.run_command("mw " + addr + " " + val) response = uboot_console.run_command("md " + addr + " 10") assert(expected_response in response) + +@pytest.mark.buildconfigspec("cmd_memory") +def test_md_repeat(uboot_console): + addr_base = "%08x" % uboot_console.config.ram_base + words = 0x10 + addr_repeat = "%08x" % (uboot_console.config.ram_base + (words * 4)) + uboot_console.run_command("md %s %x" % (addr_base, words)) + response = uboot_console.run_command("") + expected_response = addr_repeat + ": " + assert(expected_response in response)
On Tue, Nov 17, 2015 at 08:20:03PM -0700, Stephen Warren wrote:
From: Stephen Warren swarren@nvidia.com
This moves this test into the new infra-structure, and also allows it to work on real hardware, not just sandbox.
Signed-off-by: Stephen Warren swarren@nvidia.com
Reviewed-by: Tom Rini trini@konsulko.com
From: Stephen Warren swarren@nvidia.com
Most of command_ut.c simply runs shell commands which could just as easily be invoked from the command-line. Convert all of that testing to Python. I'd classify all of these tests as "system" tests since they use the standard interface to the U-Boot "system".
The remaining tests rely on features only available to C code, such as compound shell commands using \n as a separator, and validating the return value of functions that invoke shell commands directly. I'd classify these as unit tests, since they rely on mechanisms only available internally.
Signed-off-by: Stephen Warren swarren@nvidia.com --- I wonder how useful do_ut_cmd() is now. It does test some things that can't be tested from the command-line, but... --- test/command_ut.c | 133 ---------------------------------------- test/py/test_env.py | 26 +++++++- test/py/test_hush_if_test.py | 137 ++++++++++++++++++++++++++++++++++++++++++ test/py/test_shell_basics.py | 31 ++++++++++ test/py/uboot_console_base.py | 4 +- 5 files changed, 194 insertions(+), 137 deletions(-) create mode 100644 test/py/test_hush_if_test.py create mode 100644 test/py/test_shell_basics.py
diff --git a/test/command_ut.c b/test/command_ut.c index 926573a39543..35bd35ae2e30 100644 --- a/test/command_ut.c +++ b/test/command_ut.c @@ -20,21 +20,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) printf("%s: Testing commands\n", __func__); run_command("env default -f -a", 0);
- /* run a single command */ - run_command("setenv single 1", 0); - assert(!strcmp("1", getenv("single"))); - - /* make sure that compound statements work */ -#ifdef CONFIG_SYS_HUSH_PARSER - run_command("if test -n ${single} ; then setenv check 1; fi", 0); - assert(!strcmp("1", getenv("check"))); - run_command("setenv check", 0); -#endif - - /* commands separated by ; */ - run_command_list("setenv list 1; setenv list ${list}1", -1, 0); - assert(!strcmp("11", getenv("list"))); - /* commands separated by \n */ run_command_list("setenv list 1\n setenv list ${list}1", -1, 0); assert(!strcmp("11", getenv("list"))); @@ -43,11 +28,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) run_command_list("setenv list 1${list}\n", -1, 0); assert(!strcmp("111", getenv("list")));
- /* three commands in a row */ - run_command_list("setenv list 1\n setenv list ${list}2; " - "setenv list ${list}3", -1, 0); - assert(!strcmp("123", getenv("list"))); - /* a command string with \0 in it. Stuff after \0 should be ignored */ run_command("setenv list", 0); run_command_list(test_cmd, sizeof(test_cmd), 0); @@ -66,13 +46,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) assert(run_command_list("false", -1, 0) == 1); assert(run_command_list("echo", -1, 0) == 0);
- run_command("setenv foo 'setenv monty 1; setenv python 2'", 0); - run_command("run foo", 0); - assert(getenv("monty") != NULL); - assert(!strcmp("1", getenv("monty"))); - assert(getenv("python") != NULL); - assert(!strcmp("2", getenv("python"))); - #ifdef CONFIG_SYS_HUSH_PARSER run_command("setenv foo 'setenv black 1\nsetenv adder 2'", 0); run_command("run foo", 0); @@ -80,112 +53,6 @@ static int do_ut_cmd(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) assert(!strcmp("1", getenv("black"))); assert(getenv("adder") != NULL); assert(!strcmp("2", getenv("adder"))); - - /* Test the 'test' command */ - -#define HUSH_TEST(name, expr, expected_result) \ - run_command("if test " expr " ; then " \ - "setenv " #name "_" #expected_result " y; else " \ - "setenv " #name "_" #expected_result " n; fi", 0); \ - assert(!strcmp(#expected_result, getenv(#name "_" #expected_result))); \ - setenv(#name "_" #expected_result, NULL); - - /* Basic operators */ - HUSH_TEST(streq, "aaa = aaa", y); - HUSH_TEST(streq, "aaa = bbb", n); - - HUSH_TEST(strneq, "aaa != bbb", y); - HUSH_TEST(strneq, "aaa != aaa", n); - - HUSH_TEST(strlt, "aaa < bbb", y); - HUSH_TEST(strlt, "bbb < aaa", n); - - HUSH_TEST(strgt, "bbb > aaa", y); - HUSH_TEST(strgt, "aaa > bbb", n); - - HUSH_TEST(eq, "123 -eq 123", y); - HUSH_TEST(eq, "123 -eq 456", n); - - HUSH_TEST(ne, "123 -ne 456", y); - HUSH_TEST(ne, "123 -ne 123", n); - - HUSH_TEST(lt, "123 -lt 456", y); - HUSH_TEST(lt_eq, "123 -lt 123", n); - HUSH_TEST(lt, "456 -lt 123", n); - - HUSH_TEST(le, "123 -le 456", y); - HUSH_TEST(le_eq, "123 -le 123", y); - HUSH_TEST(le, "456 -le 123", n); - - HUSH_TEST(gt, "456 -gt 123", y); - HUSH_TEST(gt_eq, "123 -gt 123", n); - HUSH_TEST(gt, "123 -gt 456", n); - - HUSH_TEST(ge, "456 -ge 123", y); - HUSH_TEST(ge_eq, "123 -ge 123", y); - HUSH_TEST(ge, "123 -ge 456", n); - - HUSH_TEST(z, "-z """, y); - HUSH_TEST(z, "-z "aaa"", n); - - HUSH_TEST(n, "-n "aaa"", y); - HUSH_TEST(n, "-n """, n); - - /* Inversion of simple tests */ - HUSH_TEST(streq_inv, "! aaa = aaa", n); - HUSH_TEST(streq_inv, "! aaa = bbb", y); - - HUSH_TEST(streq_inv_inv, "! ! aaa = aaa", y); - HUSH_TEST(streq_inv_inv, "! ! aaa = bbb", n); - - /* Binary operators */ - HUSH_TEST(or_0_0, "aaa != aaa -o bbb != bbb", n); - HUSH_TEST(or_0_1, "aaa != aaa -o bbb = bbb", y); - HUSH_TEST(or_1_0, "aaa = aaa -o bbb != bbb", y); - HUSH_TEST(or_1_1, "aaa = aaa -o bbb = bbb", y); - - HUSH_TEST(and_0_0, "aaa != aaa -a bbb != bbb", n); - HUSH_TEST(and_0_1, "aaa != aaa -a bbb = bbb", n); - HUSH_TEST(and_1_0, "aaa = aaa -a bbb != bbb", n); - HUSH_TEST(and_1_1, "aaa = aaa -a bbb = bbb", y); - - /* Inversion within binary operators */ - HUSH_TEST(or_0_0_inv, "! aaa != aaa -o ! bbb != bbb", y); - HUSH_TEST(or_0_1_inv, "! aaa != aaa -o ! bbb = bbb", y); - HUSH_TEST(or_1_0_inv, "! aaa = aaa -o ! bbb != bbb", y); - HUSH_TEST(or_1_1_inv, "! aaa = aaa -o ! bbb = bbb", n); - - HUSH_TEST(or_0_0_inv_inv, "! ! aaa != aaa -o ! ! bbb != bbb", n); - HUSH_TEST(or_0_1_inv_inv, "! ! aaa != aaa -o ! ! bbb = bbb", y); - HUSH_TEST(or_1_0_inv_inv, "! ! aaa = aaa -o ! ! bbb != bbb", y); - HUSH_TEST(or_1_1_inv_inv, "! ! aaa = aaa -o ! ! bbb = bbb", y); - - setenv("ut_var_nonexistent", NULL); - setenv("ut_var_exists", "1"); - HUSH_TEST(z_varexp_quoted, "-z "$ut_var_nonexistent"", y); - HUSH_TEST(z_varexp_quoted, "-z "$ut_var_exists"", n); - setenv("ut_var_exists", NULL); - - run_command("setenv ut_var_space " "", 0); - assert(!strcmp(getenv("ut_var_space"), " ")); - run_command("setenv ut_var_test $ut_var_space", 0); - assert(!getenv("ut_var_test")); - run_command("setenv ut_var_test "$ut_var_space"", 0); - assert(!strcmp(getenv("ut_var_test"), " ")); - run_command("setenv ut_var_test " 1${ut_var_space}${ut_var_space} 2 "", 0); - assert(!strcmp(getenv("ut_var_test"), " 1 2 ")); - setenv("ut_var_space", NULL); - setenv("ut_var_test", NULL); - -#ifdef CONFIG_SANDBOX - /* File existence */ - HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", n); - run_command("sb save hostfs - creating_this_file_breaks_uboot_unit_test 0 1", 0); - HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", y); - /* Perhaps this could be replaced by an "rm" shell command one day */ - assert(!os_unlink("creating_this_file_breaks_uboot_unit_test")); - HUSH_TEST(e, "-e hostfs - creating_this_file_breaks_uboot_unit_test", n); -#endif #endif
assert(run_command("", 0) == 0); diff --git a/test/py/test_env.py b/test/py/test_env.py index 16891cd6bb15..38c305f98250 100644 --- a/test/py/test_env.py +++ b/test/py/test_env.py @@ -39,7 +39,7 @@ def unset_var(state_test_env, var): del state_test_env.env[var]
def set_var(state_test_env, var, value): - state_test_env.uboot_console.run_command("setenv " + var + " " + value) + state_test_env.uboot_console.run_command("setenv " + var + " "" + value + """) state_test_env.env[var] = value
def validate_empty(state_test_env, var): @@ -47,8 +47,10 @@ def validate_empty(state_test_env, var): assert response == ""
def validate_set(state_test_env, var, value): - response = state_test_env.uboot_console.run_command("echo $" + var) - assert response == value + # echo does not preserve leading, internal, or trailing whitespace in the + # value. printenv does, and hence allows more complete testing. + response = state_test_env.uboot_console.run_command("printenv " + var) + assert response == (var + "=" + value)
def test_env_echo_exists(state_test_env): """Echo a variable that exists""" @@ -94,3 +96,21 @@ def test_env_unset_existing(state_test_env): var = state_test_env.set_var unset_var(state_test_env, var) validate_empty(state_test_env, var) + +def test_env_expansion_spaces(state_test_env): + var_space = None + var_test = None + try: + var_space = state_test_env.get_non_existent_var() + set_var(state_test_env, var_space, " ") + + var_test = state_test_env.get_non_existent_var() + value = " 1${%(var_space)s}${%(var_space)s} 2 " % locals() + set_var(state_test_env, var_test, value) + value = " 1 2 " + validate_set(state_test_env, var_test, value) + finally: + if var_space: + unset_var(state_test_env, var_space) + if var_test: + unset_var(state_test_env, var_test) diff --git a/test/py/test_hush_if_test.py b/test/py/test_hush_if_test.py new file mode 100644 index 000000000000..3504922dcf12 --- /dev/null +++ b/test/py/test_hush_if_test.py @@ -0,0 +1,137 @@ +import os +import os.path +import pytest + +subtests = ( + # Base if functionality + + ("true", True), + ("false", False), + + # Basic operators + + ("test aaa = aaa", True), + ("test aaa = bbb", False), + + ("test aaa != bbb", True), + ("test aaa != aaa", False), + + ("test aaa < bbb", True), + ("test bbb < aaa", False), + + ("test bbb > aaa", True), + ("test aaa > bbb", False), + + ("test 123 -eq 123", True), + ("test 123 -eq 456", False), + + ("test 123 -ne 456", True), + ("test 123 -ne 123", False), + + ("test 123 -lt 456", True), + ("test 123 -lt 123", False), + ("test 456 -lt 123", False), + + ("test 123 -le 456", True), + ("test 123 -le 123", True), + ("test 456 -le 123", False), + + ("test 456 -gt 123", True), + ("test 123 -gt 123", False), + ("test 123 -gt 456", False), + + ("test 456 -ge 123", True), + ("test 123 -ge 123", True), + ("test 123 -ge 456", False), + + ("test -z """, True), + ("test -z "aaa"", False), + + ("test -n "aaa"", True), + ("test -n """, False), + + # Inversion of simple tests + + ("test ! aaa = aaa", False), + ("test ! aaa = bbb", True), + ("test ! ! aaa = aaa", True), + ("test ! ! aaa = bbb", False), + + # Binary operators + + ("test aaa != aaa -o bbb != bbb", False), + ("test aaa != aaa -o bbb = bbb", True), + ("test aaa = aaa -o bbb != bbb", True), + ("test aaa = aaa -o bbb = bbb", True), + + ("test aaa != aaa -a bbb != bbb", False), + ("test aaa != aaa -a bbb = bbb", False), + ("test aaa = aaa -a bbb != bbb", False), + ("test aaa = aaa -a bbb = bbb", True), + + # Inversion within binary operators + + ("test ! aaa != aaa -o ! bbb != bbb", True), + ("test ! aaa != aaa -o ! bbb = bbb", True), + ("test ! aaa = aaa -o ! bbb != bbb", True), + ("test ! aaa = aaa -o ! bbb = bbb", False), + + ("test ! ! aaa != aaa -o ! ! bbb != bbb", False), + ("test ! ! aaa != aaa -o ! ! bbb = bbb", True), + ("test ! ! aaa = aaa -o ! ! bbb != bbb", True), + ("test ! ! aaa = aaa -o ! ! bbb = bbb", True), + + # -z operator + + ("test -z "$ut_var_nonexistent"", True), + ("test -z "$ut_var_exists"", False), +) + +def exec_hush_if(uboot_console, expr, result): + cmd = "if " + expr + "; then echo true; else echo false; fi" + response = uboot_console.run_command(cmd) + assert response.strip() == str(result).lower() + +@pytest.mark.buildconfigspec("sys_hush_parser") +def test_hush_if_test_setup(uboot_console): + uboot_console.run_command("setenv ut_var_nonexistent") + uboot_console.run_command("setenv ut_var_exists 1") + +@pytest.mark.buildconfigspec("sys_hush_parser") +@pytest.mark.parametrize("expr,result", subtests) +def test_hush_if_test(uboot_console, expr, result): + exec_hush_if(uboot_console, expr, result) + +@pytest.mark.buildconfigspec("sys_hush_parser") +def test_hush_if_test_teardown(uboot_console): + uboot_console.run_command("setenv ut_var_exists") + +@pytest.mark.buildconfigspec("sys_hush_parser") +# We might test this on real filesystems via UMS, DFU, "save", etc. +# Of those, only UMS currently allows file removal though. +@pytest.mark.boardspec("sandbox") +def test_hush_if_test_host_file_exists(uboot_console): + test_file = uboot_console.config.result_dir + \ + "/creating_this_file_breaks_uboot_tests" + + try: + os.unlink(test_file) + except: + pass + assert not os.path.exists(test_file) + + expr = "test -e hostfs - " + test_file + exec_hush_if(uboot_console, expr, False) + + try: + with file(test_file, "wb"): + pass + assert os.path.exists(test_file) + + expr = "test -e hostfs - " + test_file + exec_hush_if(uboot_console, expr, True) + finally: + os.unlink(test_file) + + expr = "test -e hostfs - " + test_file + exec_hush_if(uboot_console, expr, False) diff --git a/test/py/test_shell_basics.py b/test/py/test_shell_basics.py new file mode 100644 index 000000000000..5477ea4e32ea --- /dev/null +++ b/test/py/test_shell_basics.py @@ -0,0 +1,31 @@ +def test_shell_execute(uboot_console): + """Test any shell command""" + response = uboot_console.run_command("echo hello") + assert response.strip() == "hello" + +def test_shell_semicolon_two(uboot_console): + """Test two shell commands separate by a semi-colon""" + cmd = "echo hello; echo world" + response = uboot_console.run_command(cmd) + # This validation method ignores the exact whitespace between the strings + assert response.index("hello") < response.index("world") + +def test_shell_semicolon_three(uboot_console): + """Test three shell commands separate by a semi-colon""" + cmd = "setenv list 1; setenv list ${list}2; setenv list ${list}3; " + \ + "echo ${list}" + response = uboot_console.run_command(cmd) + assert response.strip() == "123" + uboot_console.run_command("setenv list") + +def test_shell_run(uboot_console): + """Test the 'run' shell command""" + uboot_console.run_command("setenv foo 'setenv monty 1; setenv python 2'") + uboot_console.run_command("run foo") + response = uboot_console.run_command("echo $monty") + assert response.strip() == "1" + response = uboot_console.run_command("echo $python") + assert response.strip() == "2" + uboot_console.run_command("setenv foo") + uboot_console.run_command("setenv monty") + uboot_console.run_command("setenv python") diff --git a/test/py/uboot_console_base.py b/test/py/uboot_console_base.py index 4bf9e8303633..5ecddd88e124 100644 --- a/test/py/uboot_console_base.py +++ b/test/py/uboot_console_base.py @@ -84,7 +84,9 @@ class ConsoleBase(object): raise Exception("Bad pattern found on console: " + bad_pattern_ids[m - 1]) self.at_prompt = True - return self.p.before.strip() + # Only strip \r\n; space/TAB might be significant if testing + # indentation. + return self.p.before.strip("\r\n") except Exception as ex: self.log.error(str(ex)) self.cleanup_spawn()
participants (2)
-
Stephen Warren -
Tom Rini