Re: [PATCH v5 01/23] binman: ti-board-config: Add support for TI board config binaries

On 07.07.23 14:34, Neha Malcom Francis wrote:

The ti-board-config entry loads and validates a given YAML config file against a given schema, and generates the board config binary. K3 devices require these binaries to be packed into the final system firmware images.

Signed-off-by: Neha Malcom Francis n-francis@ti.com Reviewed-by: Simon Glass sjg@chromium.org

---

tools/binman/entries.rst | 48 ++++ tools/binman/etype/ti_board_config.py | 259 ++++++++++++++++++ tools/binman/ftest.py | 20 ++ tools/binman/test/277_ti_board_cfg.dts | 14 + .../binman/test/278_ti_board_cfg_combined.dts | 25 ++ .../binman/test/279_ti_board_cfg_no_type.dts | 11 + tools/binman/test/yaml/config.yaml | 19 ++ tools/binman/test/yaml/schema.yaml | 51 ++++ tools/binman/test/yaml/schema_notype.yaml | 40 +++ 9 files changed, 487 insertions(+) create mode 100644 tools/binman/etype/ti_board_config.py create mode 100644 tools/binman/test/277_ti_board_cfg.dts create mode 100644 tools/binman/test/278_ti_board_cfg_combined.dts create mode 100644 tools/binman/test/279_ti_board_cfg_no_type.dts create mode 100644 tools/binman/test/yaml/config.yaml create mode 100644 tools/binman/test/yaml/schema.yaml create mode 100644 tools/binman/test/yaml/schema_notype.yaml

diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index b71af801fd..14a2d03fad 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -1658,6 +1658,54 @@ by setting the size of the entry to something larger than the text.

+.. _etype_ti_board_config:

+Entry: ti-board-config: An entry containing a TI schema validated board config binary +-------------------------------------------------------------------------------------

+This etype supports generation of two kinds of board configuration +binaries: singular board config binary as well as combined board config +binary.

+Properties / Entry arguments:

• • config-file: File containing board configuration data in YAML
• • schema-file: File containing board configuration YAML schema against

•  which the config file is validated
  

+Output files:

• • board config binary: File containing board configuration binary

+These above parameters are used only when the generated binary is +intended to be a single board configuration binary. Example::

• my-ti-board-config {

•    ti-board-config {
  

•        config = "board-config.yaml";
  

•        schema = "schema.yaml";
  

•    };
  

• };

+To generate a combined board configuration binary, we pack the +needed individual binaries into a ti-board-config binary. In this case, +the available supported subnode names are board-cfg, pm-cfg, sec-cfg and +rm-cfg. The final binary is prepended with a header containing details about +the included board config binaries. Example::

• my-combined-ti-board-config {

•    ti-board-config {
  

•        board-cfg {
  

•            config = "board-cfg.yaml";
  

•            schema = "schema.yaml";
  

•        };
  

•        sec-cfg {
  

•            config = "sec-cfg.yaml";
  

•            schema = "schema.yaml";
  

•        };
  

•    }
  

• }

.. _etype_u_boot:

Entry: u-boot: U-Boot flat binary diff --git a/tools/binman/etype/ti_board_config.py b/tools/binman/etype/ti_board_config.py new file mode 100644 index 0000000000..0799e5dc59 --- /dev/null +++ b/tools/binman/etype/ti_board_config.py @@ -0,0 +1,259 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2022 Texas Instruments Incorporated - https://www.ti.com/ +# Written by Neha Malcom Francis n-francis@ti.com +# +# Entry-type module for generating schema validated TI board +# configuration binary +#

+import os +import struct +import yaml

+from collections import OrderedDict +from jsonschema import validate +from shutil import copyfileobj

+from binman.entry import Entry +from binman.etype.section import Entry_section +from dtoc import fdt_util +from u_boot_pylib import tools

+BOARDCFG = 0xB +BOARDCFG_SEC = 0xD +BOARDCFG_PM = 0xE +BOARDCFG_RM = 0xC +BOARDCFG_NUM_ELEMS = 4

+class Entry_ti_board_config(Entry_section):

• """An entry containing a TI schema validated board config binary
• This etype supports generation of two kinds of board configuration
• binaries: singular board config binary as well as combined board config
• binary.
• Properties / Entry arguments:

•    - config-file: File containing board configuration data in YAML
  

•    - schema-file: File containing board configuration YAML schema against
  

•      which the config file is validated
  

• Output files:

•    - board config binary: File containing board configuration binary
  

• These above parameters are used only when the generated binary is
• intended to be a single board configuration binary. Example::

•    my-ti-board-config {
  

•        ti-board-config {
  

•            config = "board-config.yaml";
  

•            schema = "schema.yaml";
  

•        };
  

•    };
  

• To generate a combined board configuration binary, we pack the
• needed individual binaries into a ti-board-config binary. In this case,
• the available supported subnode names are board-cfg, pm-cfg, sec-cfg and
• rm-cfg. The final binary is prepended with a header containing details about
• the included board config binaries. Example::

•    my-combined-ti-board-config {
  

•        ti-board-config {
  

•            board-cfg {
  

•                config = "board-cfg.yaml";
  

•                schema = "schema.yaml";
  

•            };
  

•            sec-cfg {
  

•                config = "sec-cfg.yaml";
  

•                schema = "schema.yaml";
  

•            };
  

•        }
  

•    }
  

• """
• def __init__(self, section, etype, node):

•    super().__init__(section, etype, node)
  

•    self._config = None
  

•    self._schema = None
  

•    self._entries = OrderedDict()
  

•    self._num_elems = BOARDCFG_NUM_ELEMS
  

•    self._fmt = '<HHHBB'
  

•    self._index = 0
  

•    self._binary_offset = 0
  

•    self._sw_rev = 1
  

•    self._devgrp = 0
  

• def ReadNode(self):

•    super().ReadNode()
  

•    self._config = fdt_util.GetString(self._node, 'config')
  

•    self._schema = fdt_util.GetString(self._node, 'schema')
  

•    # Depending on whether config file is present in node, we determine
  

•    # whether it is a combined board config binary or not
  

•    if self._config is None:
  

•        self.ReadEntries()
  

•    else:
  

•        self._config_file = tools.get_input_filename(self._config)
  

•        self._schema_file = tools.get_input_filename(self._schema)
  

• def ReadEntries(self):

•    """Read the subnodes to find out what should go in this image
  

•    """
  

•    for node in self._node.subnodes:
  

•        if 'type' not in node.props:
  

•            entry = Entry.Create(self, node, 'ti-board-config')
  

•            entry.ReadNode()
  

•            cfg_data = entry.BuildSectionData(True)
  

•            entry._cfg_data = cfg_data
  

•            self._entries[entry.name] = entry
  

•    self._num_elems = len(self._node.subnodes)
  

• def _convert_to_byte_chunk(self, val, data_type):

•    """Convert value into byte array
  

•    Args:
  

•        val: value to convert into byte array
  

•        data_type: data type used in schema, supported data types are u8,
  

•            u16 and u32
  

•    Returns:
  

•        array of bytes representing value
  

•    """
  

•    size = 0
  

•    if (data_type == '#/definitions/u8'):
  

•        size = 1
  

•    elif (data_type == '#/definitions/u16'):
  

•        size = 2
  

•    else:
  

•        size = 4
  

•    if type(val) == int:
  

•        br = val.to_bytes(size, byteorder='little')
  

•    return br
  

• def _compile_yaml(self, schema_yaml, file_yaml):

•    """Convert YAML file into byte array based on YAML schema
  

•    Args:
  

•        schema_yaml: file containing YAML schema
  

•        file_yaml: file containing config to compile
  

•    Returns:
  

•        array of bytes repesenting YAML file against YAML schema
  

•    """
  

•    br = bytearray()
  

•    for key, node in file_yaml.items():
  

•        node_schema = schema_yaml['properties'][key]
  

•        node_type = node_schema.get('type')
  

•        if not 'type' in node_schema:
  

•            br += self._convert_to_byte_chunk(node,
  

•                                            node_schema.get('$ref'))
  

•        elif node_type == 'object':
  

•            br += self._compile_yaml(node_schema, node)
  

•        elif node_type == 'array':
  

•            for item in node:
  

•                if not isinstance(item, dict):
  

•                    br += self._convert_to_byte_chunk(
  

•                        item, schema_yaml['properties'][key]['items']['$ref'])
  

•                else:
  

•                    br += self._compile_yaml(node_schema.get('items'), item)
  

•    return br
  

• def _generate_binaries(self):

•    """Generate config binary artifacts from the loaded YAML configuration file
  

•    Returns:
  

•        byte array containing config binary artifacts
  

•        or None if generation fails
  

•    """
  

•    cfg_binary = bytearray()
  

•    for key, node in self.file_yaml.items():
  

•        node_schema = self.schema_yaml['properties'][key]
  

•        br = self._compile_yaml(node_schema, node)
  

•        cfg_binary += br
  

•    return cfg_binary
  

• def _add_boardcfg(self, bcfgtype, bcfgdata):

•    """Add board config to combined board config binary
  

•    Args:
  

•        bcfgtype (int): board config type
  

•        bcfgdata (byte array): board config data
  

•    """
  

•    size = len(bcfgdata)
  

•    desc = struct.pack(self._fmt, bcfgtype,
  

•                        self._binary_offset, size, self._devgrp, 0)
  

•    with open(self.descfile, 'ab+') as desc_fh:
  

•        desc_fh.write(desc)
  

•    with open(self.bcfgfile, 'ab+') as bcfg_fh:
  

•        bcfg_fh.write(bcfgdata)
  

•    self._binary_offset += size
  

•    self._index += 1
  

• def _finalize(self):

•    """Generate final combined board config binary
  

•    Returns:
  

•        byte array containing combined board config data
  

•        or None if unable to generate
  

•    """
  

•    with open(self.descfile, 'rb') as desc_fh:
  

•        with open(self.bcfgfile, 'rb') as bcfg_fh:
  

•            with open(self.fh_file, 'ab+') as fh:
  

•                copyfileobj(desc_fh, fh)
  

•                copyfileobj(bcfg_fh, fh)
  

•    data = tools.read_file(self.fh_file)
  

•    return data
  

• def BuildSectionData(self, required):

•    if self._config is None:
  

•        self._binary_offset = 0
  

•        uniq = self.GetUniqueName()
  

•        self.fh_file = tools.get_output_filename('fh.%s' % uniq)
  

•        self.descfile = tools.get_output_filename('desc.%s' % uniq)
  

•        self.bcfgfile = tools.get_output_filename('bcfg.%s' % uniq)
  

•        # when binman runs again make sure we start clean
  

•        if os.path.exists(self.fh_file):
  

•            os.remove(self.fh_file)
  

•        if os.path.exists(self.descfile):
  

•            os.remove(self.descfile)
  

•        if os.path.exists(self.bcfgfile):
  

•            os.remove(self.bcfgfile)
  

•        with open(self.fh_file, 'wb') as f:
  

•            t_bytes = f.write(struct.pack(
  

•                '<BB', self._num_elems, self._sw_rev))
  

•        self._binary_offset += t_bytes
  

•        self._binary_offset += self._num_elems * struct.calcsize(self._fmt)
  

•        if 'board-cfg' in self._entries:
  

•            self._add_boardcfg(BOARDCFG, self._entries['board-cfg']._cfg_data)
  

•        if 'sec-cfg' in self._entries:
  

•            self._add_boardcfg(BOARDCFG_SEC, self._entries['sec-cfg']._cfg_data)
  

•        if 'pm-cfg' in self._entries:
  

•            self._add_boardcfg(BOARDCFG_PM, self._entries['pm-cfg']._cfg_data)
  

•        if 'rm-cfg' in self._entries:
  

•            self._add_boardcfg(BOARDCFG_RM, self._entries['rm-cfg']._cfg_data)
  

•        data = self._finalize()
  

•        return data
  

•    else:
  

•        with open(self._config_file, 'r') as f:
  

•            self.file_yaml = yaml.safe_load(f)
  

•        with open(self._schema_file, 'r') as sch:
  

•            self.schema_yaml = yaml.safe_load(sch)
  

•        try:
  

•            validate(self.file_yaml, self.schema_yaml)
  

•        except Exception as e:
  

•            self.Raise(f"Schema validation error: {e}")
  

•        data = self._generate_binaries()
  

•        return data
  

• def SetImagePos(self, image_pos):

•    Entry.SetImagePos(self, image_pos)
  

• def CheckEntries(self):

•    Entry.CheckEntries(self)
  

diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 43b4f850a6..b9a490a5bd 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -97,6 +97,7 @@ ENV_DATA = b'var1=1\nvar2="2"' PRE_LOAD_MAGIC = b'UBSH' PRE_LOAD_VERSION = 0x11223344.to_bytes(4, 'big') PRE_LOAD_HDR_SIZE = 0x00001000.to_bytes(4, 'big') +TI_BOARD_CONFIG_DATA = b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

# Subdirectory of the input dir to use to put test FDTs TEST_FDT_SUBDIR = 'fdts' @@ -199,6 +200,9 @@ class TestFunctional(unittest.TestCase): shutil.copytree(cls.TestFile('files'), os.path.join(cls._indir, 'files'))

•    shutil.copytree(cls.TestFile('yaml'),
  

•                    os.path.join(cls._indir, 'yaml'))
  

•    TestFunctional._MakeInputFile('compress', COMPRESS_DATA)
     TestFunctional._MakeInputFile('compress_big', COMPRESS_DATA_BIG)
     TestFunctional._MakeInputFile('bl31.bin', ATF_BL31_DATA)
  

@@ -6676,6 +6680,22 @@ fdt fdtmap Extract the devicetree blob from the fdtmap ['fit']) self.assertIn("Node '/fit': Missing tool: 'mkimage'", str(e.exception))

• def testTIBoardConfig(self):

•    """Test that a schema validated board config file can be generated"""
  

•    data = self._DoReadFile('277_ti_board_cfg.dts')
  

•    self.assertEqual(TI_BOARD_CONFIG_DATA, data)
  

• def testTIBoardConfigCombined(self):

•    """Test that a schema validated combined board config file can be generated"""
  

•    data = self._DoReadFile('278_ti_board_cfg_combined.dts')
  

•    configlen_noheader = TI_BOARD_CONFIG_DATA * 4
  

•    self.assertGreater(data, configlen_noheader)
  

• def testTIBoardConfigNoDataType(self):

•    """Test that error is thrown when data type is not supported"""
  

•    with self.assertRaises(ValueError) as e:
  

•        data = self._DoReadFile('279_ti_board_cfg_no_type.dts')
  

•    self.assertIn("Schema validation error", str(e.exception))
  
  

if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/277_ti_board_cfg.dts b/tools/binman/test/277_ti_board_cfg.dts new file mode 100644 index 0000000000..cda024c1b8 --- /dev/null +++ b/tools/binman/test/277_ti_board_cfg.dts @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0+ +/dts-v1/;

+/ {

• #address-cells = <1>;
• #size-cells = <1>;
• binman {

• ti-board-config {
  

• 	config = "yaml/config.yaml";
  

• 	schema = "yaml/schema.yaml";
  

• };
  

• };

+}; diff --git a/tools/binman/test/278_ti_board_cfg_combined.dts b/tools/binman/test/278_ti_board_cfg_combined.dts new file mode 100644 index 0000000000..95ef449cbf --- /dev/null +++ b/tools/binman/test/278_ti_board_cfg_combined.dts @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0+ +/dts-v1/;

+/ {

• binman {

• ti-board-config {
  

• 	board-cfg {
  

• 		config = "yaml/config.yaml";
  

• 		schema = "yaml/schema.yaml";
  

• 	};
  

• 	sec-cfg {
  

• 		config = "yaml/config.yaml";
  

• 		schema = "yaml/schema.yaml";
  

• 	};
  

• 	rm-cfg {
  

• 		config = "yaml/config.yaml";
  

• 		schema = "yaml/schema.yaml";
  

• 	};
  

• 	pm-cfg {
  

• 		config = "yaml/config.yaml";
  

• 		schema = "yaml/schema.yaml";
  

• 	};
  

• };
  

• };

+}; diff --git a/tools/binman/test/279_ti_board_cfg_no_type.dts b/tools/binman/test/279_ti_board_cfg_no_type.dts new file mode 100644 index 0000000000..584b7acc5a --- /dev/null +++ b/tools/binman/test/279_ti_board_cfg_no_type.dts @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0+ +/dts-v1/;

+/ {

• binman {

• ti-board-config {
  

• 	config = "yaml/config.yaml";
  

• 	schema = "yaml/schema_notype.yaml";
  

• };
  

• };

+}; diff --git a/tools/binman/test/yaml/config.yaml b/tools/binman/test/yaml/config.yaml new file mode 100644 index 0000000000..79fd67c7f4 --- /dev/null +++ b/tools/binman/test/yaml/config.yaml @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Texas Instruments Incorporated - https://www.ti.com/ +# +# Test config +# +---

+main-branch:

• obj:
• a: 0x0
• b: 0
• arr: [0, 0, 0, 0]
• another-arr:
• • #1

•  c: 0
  

•  d: 0
  

• • #2

•  c: 0
  

•  d: 0
  

diff --git a/tools/binman/test/yaml/schema.yaml b/tools/binman/test/yaml/schema.yaml new file mode 100644 index 0000000000..60bf56f671 --- /dev/null +++ b/tools/binman/test/yaml/schema.yaml @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Texas Instruments Incorporated - https://www.ti.com/ +# +# Test schema +# +---

+definitions:

• u8:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xff
  

• u16:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xffff
  

• u32:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xffffffff
  

+type: object +properties:

• main-branch:

•    type: object
  

•    properties:
  

•        obj:
  

•            type: object
  

•            properties:
  

•                a:
  

•                    $ref: "#/definitions/u32"
  

•                b:
  

•                    $ref: "#/definitions/u16"
  

•        arr:
  

•            type: array
  

•            minItems: 4
  

•            maxItems: 4
  

•            items:
  

•                $ref: "#/definitions/u8"
  

•        another-arr:
  

•            type: array
  

•            minItems: 2
  

•            maxItems: 2
  

•            items:
  

•                type: object
  

•                properties:
  

•                    c:
  

•                        $ref: "#/definitions/u8"
  

•                    d:
  

•                        $ref: "#/definitions/u8"
  

diff --git a/tools/binman/test/yaml/schema_notype.yaml b/tools/binman/test/yaml/schema_notype.yaml new file mode 100644 index 0000000000..d45d6cdf7e --- /dev/null +++ b/tools/binman/test/yaml/schema_notype.yaml @@ -0,0 +1,40 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (C) 2022 Texas Instruments Incorporated - https://www.ti.com/ +# +# Test schema +# +---

+definitions:

• u8:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xff
  

• u16:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xffff
  

• u32:

•    type: integer
  

•    minimum: 0
  

•    maximum: 0xffffffff
  

+type: object +properties:

• main-branch:

•    type: object
  

•    properties:
  

•        obj:
  

•            type: object
  

•            properties:
  

•                a:
  

•                    $ref: "#/definitions/u4"
  

•                b:
  

•                    $ref: "#/definitions/u16"
  

•        arr:
  

•            type: array
  

•            minItems: 4
  

•            maxItems: 4
  

•            items:
  

•                $ref: "#/definitions/u8"
  

... Applying: binman: ti-board-config: Add support for TI board config binaries .git/rebase-apply/patch:538: new blank line at EOF. + .git/rebase-apply/patch:584: new blank line at EOF. + warning: 2 lines add whitespace errors.

Jan