[PATCH] test/py: tpm2: Skip tpm pytest based on env variable
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com ---
test/py/tests/test_tpm2.py | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py index 70f906da51..bb147d4e88 100644 --- a/test/py/tests/test_tpm2.py +++ b/test/py/tests/test_tpm2.py @@ -18,6 +18,15 @@ any password. * Commands like pcr_setauthpolicy and pcr_resetauthpolicy are not implemented here because they would fail the tests in most cases (TPMs do not implement them and return an error). + + +Note: +This test doesn't rely on boardenv_* configuration value but can change test +behavior. + +* Setup env__tpm_device_test_skip to True if tests with TPM devices should be +skipped. + """
updates = 0 @@ -29,6 +38,9 @@ def force_init(u_boot_console, force=False): twice will spawn an error used to detect that the TPM was not reset and no initialization code should be run. """ + skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') output = u_boot_console.run_command('tpm2 init') if force or not 'Error' in output: u_boot_console.run_command('echo --- start of init ---') @@ -44,6 +56,10 @@ def force_init(u_boot_console, force=False): def test_tpm2_init(u_boot_console): """Init the software stack to use TPMv2 commands."""
+ skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') + u_boot_console.run_command('tpm2 init') output = u_boot_console.run_command('echo $?') assert output.endswith('0') @@ -55,6 +71,9 @@ def test_tpm2_startup(u_boot_console): Initiate the TPM internal state machine. """
+ skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR') output = u_boot_console.run_command('echo $?') assert output.endswith('0') @@ -66,6 +85,9 @@ def test_tpm2_self_test_full(u_boot_console): Ask the TPM to perform all self tests to also enable full capabilities. """
+ skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') u_boot_console.run_command('tpm2 self_test full') output = u_boot_console.run_command('echo $?') assert output.endswith('0') @@ -78,6 +100,9 @@ def test_tpm2_continue_self_test(u_boot_console): to enter a fully operational state. """
+ skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') u_boot_console.run_command('tpm2 self_test continue') output = u_boot_console.run_command('echo $?') assert output.endswith('0') @@ -95,6 +120,9 @@ def test_tpm2_clear(u_boot_console): PLATFORM hierarchies are also available. """
+ skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False) + if skip_test: + pytest.skip('skip TPM device test') u_boot_console.run_command('tpm2 clear TPM2_RH_LOCKOUT') output = u_boot_console.run_command('echo $?') assert output.endswith('0')
Hi, Any update on this patch?
Thanks T Karthik
-----Original Message----- From: Ashok Reddy Soma ashok.reddy.soma@xilinx.com Sent: Friday, July 23, 2021 5:48 PM To: u-boot@lists.denx.de Cc: miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com; Ashok Reddy Soma ashokred@xilinx.com Subject: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
test/py/tests/test_tpm2.py | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py index 70f906da51..bb147d4e88 100644 --- a/test/py/tests/test_tpm2.py +++ b/test/py/tests/test_tpm2.py @@ -18,6 +18,15 @@ any password.
- Commands like pcr_setauthpolicy and pcr_resetauthpolicy are not
implemented here because they would fail the tests in most cases (TPMs do not implement them and return an error).
+Note: +This test doesn't rely on boardenv_* configuration value but can change +test behavior.
+* Setup env__tpm_device_test_skip to True if tests with TPM devices +should be skipped.
"""
updates = 0 @@ -29,6 +38,9 @@ def force_init(u_boot_console, force=False): twice will spawn an error used to detect that the TPM was not reset and no initialization code should be run. """
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')+56,10 @@ def force_init(u_boot_console, force=False): def test_tpm2_init(u_boot_console): """Init the software stack to use TPMv2 commands."""
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')- u_boot_console.run_command('tpm2 init') output = u_boot_console.run_command('echo $?') assert output.endswith('0')
@@ -55,6 +71,9 @@ def test_tpm2_startup(u_boot_console): Initiate the TPM internal state machine. """
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')@@ -66,6 +85,9 @@ def test_tpm2_self_test_full(u_boot_console): Ask the TPM to perform all self tests to also enable full capabilities. """
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')@@ -78,6 +100,9 @@ def test_tpm2_continue_self_test(u_boot_console): to enter a fully operational state. """
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')@@ -95,6 +120,9 @@ def test_tpm2_clear(u_boot_console): PLATFORM hierarchies are also available. """
- skip_test =
u_boot_console.config.env.get('env__tpm_device_test_skip', False)
- if skip_test:
pytest.skip('skip TPM device test')-- 2.17.1
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from the configuration file. I referred test/py/tests/test_sleep.py for writing this test case, hence copied comments as well. Should I change this NOTE in comment section?
Thanks T Karthik
-- Tom
On Thu, Aug 05, 2021 at 05:56:50AM +0000, T Karthik Reddy wrote:
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from the configuration file. I referred test/py/tests/test_sleep.py for writing this test case, hence copied comments as well. Should I change this NOTE in comment section?
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
Hi Tom,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Thursday, August 5, 2021 6:53 PM To: T Karthik Reddy tkarthik@xilinx.com Cc: Ashok Reddy Soma ashokred@xilinx.com; u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Thu, Aug 05, 2021 at 05:56:50AM +0000, T Karthik Reddy wrote:
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from
the configuration file.
I referred test/py/tests/test_sleep.py for writing this test case, hence
copied comments as well.
Should I change this NOTE in comment section?
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
I see this patch is not yet applied on the repo, could you please let me know when this will be applied?
Thanks T karthik
-- Tom
On Fri, Sep 17, 2021 at 11:49:51AM +0000, T Karthik Reddy wrote:
Hi Tom,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Thursday, August 5, 2021 6:53 PM To: T Karthik Reddy tkarthik@xilinx.com Cc: Ashok Reddy Soma ashokred@xilinx.com; u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Thu, Aug 05, 2021 at 05:56:50AM +0000, T Karthik Reddy wrote:
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from
the configuration file.
I referred test/py/tests/test_sleep.py for writing this test case, hence
copied comments as well.
Should I change this NOTE in comment section?
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
I see this patch is not yet applied on the repo, could you please let me know when this will be applied?
Ilias, since you've said you'd review TPM related patches, does this: https://patchwork.ozlabs.org/project/uboot/patch/1624340962-91190-1-git-send... work for you? Thanks.
Hi Tom,
On Fri, 17 Sept 2021 at 15:08, Tom Rini trini@konsulko.com wrote:
On Fri, Sep 17, 2021 at 11:49:51AM +0000, T Karthik Reddy wrote:
Hi Tom,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Thursday, August 5, 2021 6:53 PM To: T Karthik Reddy tkarthik@xilinx.com Cc: Ashok Reddy Soma ashokred@xilinx.com; u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Thu, Aug 05, 2021 at 05:56:50AM +0000, T Karthik Reddy wrote:
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from
the configuration file.
I referred test/py/tests/test_sleep.py for writing this test case, hence
copied comments as well.
Should I change this NOTE in comment section?
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
I see this patch is not yet applied on the repo, could you please let me know when this will be applied?
Ilias, since you've said you'd review TPM related patches, does this: https://patchwork.ozlabs.org/project/uboot/patch/1624340962-91190-1-git-send... work for you? Thanks.
I am not too familiar with the pytest framework we have, but couldn't we do this automatically? IOW instead of explicitly adding a variable, do a 'tpm info' and if that doesn't return a valid device, skip the tests.
Regards /Ilias
-- Tom
Hi
On 9/18/21 11:48 AM, Ilias Apalodimas wrote:
Hi Tom,
On Fri, 17 Sept 2021 at 15:08, Tom Rini trini@konsulko.com wrote:
On Fri, Sep 17, 2021 at 11:49:51AM +0000, T Karthik Reddy wrote:
Hi Tom,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Thursday, August 5, 2021 6:53 PM To: T Karthik Reddy tkarthik@xilinx.com Cc: Ashok Reddy Soma ashokred@xilinx.com; u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Thu, Aug 05, 2021 at 05:56:50AM +0000, T Karthik Reddy wrote:
Hello Tom Rini,
-----Original Message----- From: Tom Rini trini@konsulko.com Sent: Wednesday, August 4, 2021 8:52 PM To: Ashok Reddy Soma ashokred@xilinx.com Cc: u-boot@lists.denx.de; miquel.raynal@bootlin.com; git git@xilinx.com; T Karthik Reddy tkarthik@xilinx.com Subject: Re: [PATCH] test/py: tpm2: Skip tpm pytest based on env variable
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
> From: T Karthik Reddy t.karthik.reddy@xilinx.com > > Tpm test cases relies on tpm device setup. Provide an environment > variable "env__tpm_device_test_skip = True" to skip the test case > if tpm device is not present. > Only needed will have to add variable to the py-test framework. > Test runs successfully even this variable is absent. > > Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com > Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com
Why is this not using boardenv_* like other tests do to disable running?
What I meant here was this test doesn't depend on configuration value from boardenv_*. But it uses env__tpm_device_test_skip variable from
the configuration file.
I referred test/py/tests/test_sleep.py for writing this test case, hence
copied comments as well.
Should I change this NOTE in comment section?
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
I see this patch is not yet applied on the repo, could you please let me know when this will be applied?
Ilias, since you've said you'd review TPM related patches, does this: https://patchwork.ozlabs.org/project/uboot/patch/1624340962-91190-1-git-send... work for you? Thanks.
I am not too familiar with the pytest framework we have, but couldn't we do this automatically? IOW instead of explicitly adding a variable, do a 'tpm info' and if that doesn't return a valid device, skip the tests.
Unfortunately not. Because imagine how this will work if you have tpm on board but there is bug in the driver that it is not probed. Then tpm info will return no device and all tests will be silently skipped.
It would be much better it the test itself was designed from the beginning that you need to state that board has TPM. But it wasn't that's why we designed this patch in a way that you declare that you want to skip these tests which is OK.
It doesn't make any sense to go back and start to have multiple defconfigs for different configurations where all boards works quite well with one defconfig. That's why we enable TPM because it on SOM but we need to be able to disable these tests on others.
Thanks, Michal
Hi Michal,
[...]
Thanks for explaining. This got me to re-read the existing tests and while none of it is as clear as I would like, your example here is just as clear as the rest, so it's fine and I'll pick it up soon.
I see this patch is not yet applied on the repo, could you please let me know when this will be applied?
Ilias, since you've said you'd review TPM related patches, does this: https://patchwork.ozlabs.org/project/uboot/patch/1624340962-91190-1-git-send... work for you? Thanks.
I am not too familiar with the pytest framework we have, but couldn't we do this automatically? IOW instead of explicitly adding a variable, do a 'tpm info' and if that doesn't return a valid device, skip the tests.
Unfortunately not. Because imagine how this will work if you have tpm on board but there is bug in the driver that it is not probed. Then tpm info will return no device and all tests will be silently skipped.
Looking at tpm_info I think it's the other way around, isn't it ? tpm_info will call get_tpm(), which in theory will fail if there's no TPM probed or not present. So even in that case the self-tests will fail.
OTOH the 'tpm info' command just returns a CMD Success/Failure, so it's not easy to figure out if a TPM is indeed present or not without refactoring the cmdline interface. So I think this is reasonable for now
Acked-by: Ilias Apalodimas ilias.apalodimas@linaro.org
It would be much better it the test itself was designed from the beginning that you need to state that board has TPM. But it wasn't that's why we designed this patch in a way that you declare that you want to skip these tests which is OK.
It doesn't make any sense to go back and start to have multiple defconfigs for different configurations where all boards works quite well with one defconfig. That's why we enable TPM because it on SOM but we need to be able to disable these tests on others.
Thanks, Michal
On Fri, Jul 23, 2021 at 06:18:26AM -0600, Ashok Reddy Soma wrote:
From: T Karthik Reddy t.karthik.reddy@xilinx.com
Tpm test cases relies on tpm device setup. Provide an environment variable "env__tpm_device_test_skip = True" to skip the test case if tpm device is not present. Only needed will have to add variable to the py-test framework. Test runs successfully even this variable is absent.
Signed-off-by: T Karthik Reddy t.karthik.reddy@xilinx.com Signed-off-by: Ashok Reddy Soma ashok.reddy.soma@xilinx.com Acked-by: Ilias Apalodimas ilias.apalodimas@linaro.org
Applied to u-boot/master, thanks!
participants (5)
-
Ashok Reddy Soma -
Ilias Apalodimas -
Michal Simek -
T Karthik Reddy -
Tom Rini