[RFC] Proposed location to host the firmware handoff specification.
Hi,
Arm worked to draft a firmware handoff [1] specification, evolving it based on community feedback. This activity followed the request of some members of the Arm ecosystem [2]. The spec (still at ALP – feedback/comments welcome!) standardizes how information is propagated between different firmware components during boot.
The spec hopes to remove the reliance on bespoke/platform-specific information handoff mechanisms, thus reducing the code maintenance burden.
The concept of entry types is present in the spec – these are data structure layouts that carry a specific type of data. New types are meant to be added, following the needs and use-cases of the different communities. Thus, these communities should be empowered to request new types!
To enable community contributions, the specification must be hosted in a location that is friendly to change requests.
We propose to host the spec in trustedfirmware.org (tf.org).
Tf.org hosts several open-source projects and already has an open governance model. TF-A, and the associated community, rely on tf.org, and thus are already well equipped to maintain this specification and keep it up to date. Tf.org is agnostic of any downstream projects that would adopt this specification (e.g. U-boot, EDK2, etc.).
We welcome the views of the communities and want to understand if there are any strong objections to what’s being proposed! If anyone has objections, we are happy to consider alternatives and associated trade-offs.
Regards
[1] https://developer.arm.com/documentation/den0135/latest [2] Re: [TF-A] Proposal: TF-A to adopt hand-off blocks (HOBs) for information passing between boot stages - TF-A - lists.trustedfirmware.orghttps://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.org/thread/UZ3H3BVVEIAH6BGFMKFPGIIMS2IZNQKO/#KGATZYDHQWDRPA6ABHQUEB6JJUEGMAMO
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
If that is not acceptable then we should look for something else.
Regards, Simon
On Thu, 23 Jun 2022 at 08:20, Jose Marinho Jose.Marinho@arm.com wrote:
Hi,
Arm worked to draft a firmware handoff [1] specification, evolving it based on community feedback.
This activity followed the request of some members of the Arm ecosystem [2]. The spec (still at ALP – feedback/comments welcome!) standardizes how information is propagated between different firmware components during boot.
The spec hopes to remove the reliance on bespoke/platform-specific information handoff mechanisms, thus reducing the code maintenance burden.
The concept of entry types is present in the spec – these are data structure layouts that carry a specific type of data. New types are meant to be added, following the needs and use-cases of the different communities. Thus, these communities should be empowered to request new types!
To enable community contributions, the specification must be hosted in a location that is friendly to change requests.
We propose to host the spec in trustedfirmware.org (tf.org).
Tf.org hosts several open-source projects and already has an open governance model. TF-A, and the associated community, rely on tf.org, and thus are already well equipped to maintain this specification and keep it up to date. Tf.org is agnostic of any downstream projects that would adopt this specification (e.g. U-boot, EDK2, etc.).
We welcome the views of the communities and want to understand if there are any strong objections to what’s being proposed! If anyone has objections, we are happy to consider alternatives and associated trade-offs.
Regards
[1] https://developer.arm.com/documentation/den0135/latest
[2] Re: [TF-A] Proposal: TF-A to adopt hand-off blocks (HOBs) for information passing between boot stages - TF-A - lists.trustedfirmware.org
Hi all,
Thanks for pushing on this
On Thu, 30 Jun 2022 at 12:24, Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
In my mind this effort is similar to EBBR since the goal is to be architecture agnostic.
I am not too picky on the location myself as long as it's open and other people can contribute and propose changes.
Regards /Ilias
If that is not acceptable then we should look for something else.
Regards, Simon
On Thu, 23 Jun 2022 at 08:20, Jose Marinho Jose.Marinho@arm.com wrote:
Hi,
Arm worked to draft a firmware handoff [1] specification, evolving it based on community feedback.
This activity followed the request of some members of the Arm ecosystem [2]. The spec (still at ALP – feedback/comments welcome!) standardizes how information is propagated between different firmware components during boot.
The spec hopes to remove the reliance on bespoke/platform-specific information handoff mechanisms, thus reducing the code maintenance burden.
The concept of entry types is present in the spec – these are data structure layouts that carry a specific type of data. New types are meant to be added, following the needs and use-cases of the different communities. Thus, these communities should be empowered to request new types!
To enable community contributions, the specification must be hosted in a location that is friendly to change requests.
We propose to host the spec in trustedfirmware.org (tf.org).
Tf.org hosts several open-source projects and already has an open governance model. TF-A, and the associated community, rely on tf.org, and thus are already well equipped to maintain this specification and keep it up to date. Tf.org is agnostic of any downstream projects that would adopt this specification (e.g. U-boot, EDK2, etc.).
We welcome the views of the communities and want to understand if there are any strong objections to what’s being proposed! If anyone has objections, we are happy to consider alternatives and associated trade-offs.
Regards
[1] https://developer.arm.com/documentation/den0135/latest
[2] Re: [TF-A] Proposal: TF-A to adopt hand-off blocks (HOBs) for information passing between boot stages - TF-A - lists.trustedfirmware.org
On Thu, Jun 30, 2022 at 01:40:11PM +0300, Ilias Apalodimas wrote:
Hi all,
Thanks for pushing on this
On Thu, 30 Jun 2022 at 12:24, Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
In my mind this effort is similar to EBBR since the goal is to be architecture agnostic.
I am not too picky on the location myself as long as it's open and other people can contribute and propose changes.
I agree with this sentiment. And to the extent that being distributed is a good thing (and I believe it is), a good governance model and ease of contribution matters more-so than hosting location and to that end, git.trustedfirmware.org / reviews.trustedfirmware.org (which I see lets you use GitHub to sign-in/auth) makes sense to me.
Hi Simon
A couple of important points: * TF-A does not aim to replace U-Boot SPL; it aims to reduce fragmentation in secure firmware. The permissive license may not be ideal but from an OSS PoV, but it enables much wider adoption than other licensing schemes. But that's perhaps a philosophical discussion for another time! * TF-A is only one project at https://www.trustedfirmware.org/. They are all under open governance. If the spec were hosted there, it would be in a separate project to TF-A, with separate maintainers.
Having said that, if you're still not comfortable with https://www.trustedfirmware.org/ as a location for the spec, we will consider alternatives. I'm not sure whether http://devicetree.org/ would be acceptable to all infrastructure partners (e.g. Harb - do you have an opinion?). We'd also need to get agreement from those maintainers. Ilias suggested a model similar to EBBR. That would work for us too but please note, this would mean the project being Arm governed (with external maintainers if needed) and hosted at https://gitlab.arm.com/, since Arm's OSS group no longer uses https://github.com/ARM-software.
Regards
Dan.
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 30 June 2022 10:25
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree- org
If that is not acceptable then we should look for something else.
Regards, Simon
On Thu, 23 Jun 2022 at 08:20, Jose Marinho Jose.Marinho@arm.com wrote:
Hi,
Arm worked to draft a firmware handoff [1] specification, evolving it based
on community feedback.
This activity followed the request of some members of the Arm ecosystem
[2].
The spec (still at ALP – feedback/comments welcome!) standardizes how
information is propagated between different firmware components during boot.
The spec hopes to remove the reliance on bespoke/platform-specific
information handoff mechanisms, thus reducing the code maintenance burden.
The concept of entry types is present in the spec – these are data structure
layouts that carry a specific type of data.
New types are meant to be added, following the needs and use-cases of
the different communities.
Thus, these communities should be empowered to request new types!
To enable community contributions, the specification must be hosted in a
location that is friendly to change requests.
We propose to host the spec in trustedfirmware.org (tf.org).
Tf.org hosts several open-source projects and already has an open
governance model.
TF-A, and the associated community, rely on tf.org, and thus are already
well equipped to maintain this specification and keep it up to date.
Tf.org is agnostic of any downstream projects that would adopt this
specification (e.g. U-boot, EDK2, etc.).
We welcome the views of the communities and want to understand if
there are any strong objections to what’s being proposed!
If anyone has objections, we are happy to consider alternatives and
associated trade-offs.
Regards
[1] https://developer.arm.com/documentation/den0135/latest
[2] Re: [TF-A] Proposal: TF-A to adopt hand-off blocks (HOBs) for information passing between boot stages - TF-A - lists.trustedfirmware.org
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Rob
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
Regards, Simon
On Tue, Jul 5, 2022 at 10:37 AM Simon Glass sjg@chromium.org wrote:
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
That's a stretch IMO. Perhaps if this was a new output (DTB) format for easier parsing, I'd agree. It's related to DT only as much as any other data passed between 2 boot components (remember ATAGS?).
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
It seems to me that TF is the main driver and user of this, so I don't see the issue with them hosting it at least to start as long as there's not barriers to contributions. It's just a namespace like devicetree-org. Personally, I'd be more concerned on what the source format is (I assume the plan is not to commit PDFs) and what the output generation is. GH has a lot of nice features to support that which we've used for the DT spec and EBBR.
I'm not saying no to devicetree-org either. If the consensus is to put it there, I really don't care so much as it takes less time to create a new repo there than to write this email.
Rob
Hi Rob,
On Tue, 5 Jul 2022 at 11:27, Rob Herring robh@kernel.org wrote:
On Tue, Jul 5, 2022 at 10:37 AM Simon Glass sjg@chromium.org wrote:
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
That's a stretch IMO. Perhaps if this was a new output (DTB) format for easier parsing, I'd agree. It's related to DT only as much as any other data passed between 2 boot components (remember ATAGS?).
Yes it is a stretch. I'm just making the case.
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
It seems to me that TF is the main driver and user of this, so I don't see the issue with them hosting it at least to start as long as there's not barriers to contributions. It's just a namespace like devicetree-org. Personally, I'd be more concerned on what the source format is (I assume the plan is not to commit PDFs) and what the output generation is. GH has a lot of nice features to support that which we've used for the DT spec and EBBR.
Yes the DT spec works well and I hope the same thing can be used.
I'm not saying no to devicetree-org either. If the consensus is to put it there, I really don't care so much as it takes less time to create a new repo there than to write this email.
I do hope that this can become a standard beyond ARM, e.g. with Intel and another i can think of. Intel is essentially trying to create a different thing independently, although has apparently adjusted to use device tree due to its self-describing properties. I suspect that having this spec in an ARM site would be a barrier to that.
I am OK with ARM TF being the means to get this into the open, but not with it being the final destination.
If we cannot agree on anything better, I am happy with creating a new project on github. We'll need to pick someone to own it and make final calls when there is disagreement.
Regards, Simon
Hi All,
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 06 July 2022 09:34 To: Rob Herring robh@kernel.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer El- Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: Re: [RFC] Proposed location to host the firmware handoff specification.
Hi Rob,
On Tue, 5 Jul 2022 at 11:27, Rob Herring robh@kernel.org wrote:
On Tue, Jul 5, 2022 at 10:37 AM Simon Glass sjg@chromium.org wrote:
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org
wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
That's a stretch IMO. Perhaps if this was a new output (DTB) format for easier parsing, I'd agree. It's related to DT only as much as any other data passed between 2 boot components (remember ATAGS?).
Yes it is a stretch. I'm just making the case.
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
It seems to me that TF is the main driver and user of this, so I don't see the issue with them hosting it at least to start as long as there's not barriers to contributions. It's just a namespace like devicetree-org. Personally, I'd be more concerned on what the source format is (I assume the plan is not to commit PDFs) and what the output generation is. GH has a lot of nice features to support that which we've used for the DT spec and EBBR.
The default working plan is for the source format to be sphinx. Other alternatives/suggestions are welcome.
The output generated should be html (pdf can be supported too for 0/negligible effort). This generation process can and will most likely evolve over time, depending on community direction/desire and the tools we have at our disposal. The process should be as automated as possible given any practical constraints 😊 .
Regards, Jose
Yes the DT spec works well and I hope the same thing can be used.
I'm not saying no to devicetree-org either. If the consensus is to put it there, I really don't care so much as it takes less time to create a new repo there than to write this email.
I do hope that this can become a standard beyond ARM, e.g. with Intel and another i can think of. Intel is essentially trying to create a different thing independently, although has apparently adjusted to use device tree due to its self-describing properties. I suspect that having this spec in an ARM site would be a barrier to that.
I am OK with ARM TF being the means to get this into the open, but not with it being the final destination.
If we cannot agree on anything better, I am happy with creating a new project on github. We'll need to pick someone to own it and make final calls when there is disagreement.
Regards, Simon
Hi,
My concern with a standalone gitlab project is that it might be treated as an experiment by the wider community and be neglected. Do we have any gitlab projects (tf.org or otherwise) that have some "official" value?
@Jose, do you see the location getting used for all future spec reviews? If yes, then the gitlab project will acquire some legitimacy over time and we can start with it now. But if we see this as a one off, then I would prefer the spec being hosted on tf.org.
For the actual mechanics, I prefer reviewing docs and later updates as fragments instead of looking at the file and a separate changelog. So a platform (gerrit, gitlab, etc) that facilitates the approach would be helpful IMO.
-Varun
-----Original Message----- From: Jose Marinho via TF-A tf-a@lists.trustedfirmware.org Sent: Wednesday, 6 July 2022 11:50 AM To: Simon Glass sjg@chromium.org; Rob Herring robh@kernel.org Cc: tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer El-Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: [TF-A] Re: [RFC] Proposed location to host the firmware handoff specification.
External email: Use caution opening links or attachments
Hi All,
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 06 July 2022 09:34 To: Rob Herring robh@kernel.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer El- Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: Re: [RFC] Proposed location to host the firmware handoff specification.
Hi Rob,
On Tue, 5 Jul 2022 at 11:27, Rob Herring robh@kernel.org wrote:
On Tue, Jul 5, 2022 at 10:37 AM Simon Glass sjg@chromium.org wrote:
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org
wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
That's a stretch IMO. Perhaps if this was a new output (DTB) format for easier parsing, I'd agree. It's related to DT only as much as any other data passed between 2 boot components (remember ATAGS?).
Yes it is a stretch. I'm just making the case.
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
It seems to me that TF is the main driver and user of this, so I don't see the issue with them hosting it at least to start as long as there's not barriers to contributions. It's just a namespace like devicetree-org. Personally, I'd be more concerned on what the source format is (I assume the plan is not to commit PDFs) and what the output generation is. GH has a lot of nice features to support that which we've used for the DT spec and EBBR.
The default working plan is for the source format to be sphinx. Other alternatives/suggestions are welcome.
The output generated should be html (pdf can be supported too for 0/negligible effort). This generation process can and will most likely evolve over time, depending on community direction/desire and the tools we have at our disposal. The process should be as automated as possible given any practical constraints 😊 .
Regards, Jose
Yes the DT spec works well and I hope the same thing can be used.
I'm not saying no to devicetree-org either. If the consensus is to put it there, I really don't care so much as it takes less time to create a new repo there than to write this email.
I do hope that this can become a standard beyond ARM, e.g. with Intel and another i can think of. Intel is essentially trying to create a different thing independently, although has apparently adjusted to use device tree due to its self-describing properties. I suspect that having this spec in an ARM site would be a barrier to that.
I am OK with ARM TF being the means to get this into the open, but not with it being the final destination.
If we cannot agree on anything better, I am happy with creating a new project on github. We'll need to pick someone to own it and make final calls when there is disagreement.
Regards, Simon
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Varun,
The intent is for the proposed location (a new repository hosted in tf.org) to be used to review proposed changes to this spec.
The details of the contribution and review process will be debated in the ML once we reach consensus on the hosting location.
Regards, Jose
-----Original Message----- From: Varun Wadekar vwadekar@nvidia.com Sent: 06 July 2022 12:58 To: Jose Marinho Jose.Marinho@arm.com; Simon Glass sjg@chromium.org; Rob Herring robh@kernel.org Cc: tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot- architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer El- Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: RE: [RFC] Proposed location to host the firmware handoff specification.
Hi,
My concern with a standalone gitlab project is that it might be treated as an experiment by the wider community and be neglected. Do we have any gitlab projects (tf.org or otherwise) that have some "official" value?
@Jose, do you see the location getting used for all future spec reviews? If yes, then the gitlab project will acquire some legitimacy over time and we can start with it now. But if we see this as a one off, then I would prefer the spec being hosted on tf.org.
For the actual mechanics, I prefer reviewing docs and later updates as fragments instead of looking at the file and a separate changelog. So a platform (gerrit, gitlab, etc) that facilitates the approach would be helpful IMO.
-Varun
-----Original Message----- From: Jose Marinho via TF-A tf-a@lists.trustedfirmware.org Sent: Wednesday, 6 July 2022 11:50 AM To: Simon Glass sjg@chromium.org; Rob Herring robh@kernel.org Cc: tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot- architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer El- Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: [TF-A] Re: [RFC] Proposed location to host the firmware handoff specification.
External email: Use caution opening links or attachments
Hi All,
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 06 July 2022 09:34 To: Rob Herring robh@kernel.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; François Ozog francois.ozog@linaro.org; Manish Pandey2
Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Samer
El-
Haj-Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: Re: [RFC] Proposed location to host the firmware handoff specification.
Hi Rob,
On Tue, 5 Jul 2022 at 11:27, Rob Herring robh@kernel.org wrote:
On Tue, Jul 5, 2022 at 10:37 AM Simon Glass sjg@chromium.org wrote:
Hi Rob,
On Tue, 5 Jul 2022 at 09:24, Rob Herring robh@kernel.org wrote:
On Thu, Jun 30, 2022 at 3:24 AM Simon Glass sjg@chromium.org
wrote:
Hi Jose,
I don't think this is correct. TF-A is a project that aims to replace U-Boot SPL (and perhaps other components) with more closed firmware, e.g. the permissive license.
This spec needs to be in a neutral place, not captive of one project.
Given its close relationship to device tree, I suggest github.com/devicetree-org
The only relationship to DT I see is DT is a payload as is ACPI. So I don't think dt.org is the right place.
Actually I was about to email you about this. Here's how I see it.
DT is a base structure to allow self-describing data to be stored. This is in contrast with ACPI where there is just a header, but it is not possible to read the data without specific parsing code for the particular table types. Let's ignore ACPI for this discussion.
Unfortunately DT has an overhead and is too expensive for early firmware use. It takes 3-4KB of code for libfdt as well as extra code and data to read properties, etc.
Transfer List aims to bridge the gap, allowing simple C structures to be put into a tagged data structure. The intention is that anything more complex than that would use DT.
So there is at least some relationship: simple stuff = Transfer list, complex stuff = DT.
That's a stretch IMO. Perhaps if this was a new output (DTB) format for easier parsing, I'd agree. It's related to DT only as much as any other data passed between 2 boot components (remember
ATAGS?).
Yes it is a stretch. I'm just making the case.
The Transfer List spec specifies the data format for each entry type (the analog to the schema). The DT provides the format and schema for more complicated stuff.
We could perhaps put it in an entirely separate repo, but to me there is a relationship with DT.
It seems to me that TF is the main driver and user of this, so I don't see the issue with them hosting it at least to start as long as there's not barriers to contributions. It's just a namespace like devicetree-org. Personally, I'd be more concerned on what the source format is (I assume the plan is not to commit PDFs) and what the output generation is. GH has a lot of nice features to support that which we've used for the DT spec and EBBR.
The default working plan is for the source format to be sphinx. Other alternatives/suggestions are welcome.
The output generated should be html (pdf can be supported too for 0/negligible effort). This generation process can and will most likely evolve over time, depending on community direction/desire and the tools we have at our disposal. The process should be as automated as possible given any practical constraints 😊 .
Regards, Jose
Yes the DT spec works well and I hope the same thing can be used.
I'm not saying no to devicetree-org either. If the consensus is to put it there, I really don't care so much as it takes less time to create a new repo there than to write this email.
I do hope that this can become a standard beyond ARM, e.g. with Intel and another i can think of. Intel is essentially trying to create a different thing independently, although has apparently adjusted to use device tree due to its self-describing properties. I suspect that having this spec in an ARM site would be a barrier to that.
I am OK with ARM TF being the means to get this into the open, but not with it being the final destination.
If we cannot agree on anything better, I am happy with creating a new project on github. We'll need to pick someone to own it and make final calls when there is disagreement.
Regards, Simon
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Jose,
Can you provide a bit more background on where this proposal suddenly came from and what considerations went into its design? The mailing list discussion from last year sort of petered out without much concrete agreement, and now you're here with a full specification. It's hard to provide feedback on a finished document that doesn't include any justification for the design choices it makes and without knowing what concrete requirements may have influenced it.
With that caveat (of not knowing why you made the choices you did and what good reasons you may have had), here are the main concerns I came up with on a first read:
1. Where did the requirement for 16-byte alignment come from? It seems a bit excessive considering that no data field is actually larger than 4 bytes (and it looks like it influenced several other choices in the spec, e.g. unnecessarily large transfer entry header because the bytes to the alignment boundary have to be filled anyway). For something that's supposed to be the "lightweight" alternative to more elaborate formats, this design doesn't really feel that lightweight anymore.
2. The table entry header seems unnecessarily large. What's the point of including a "header length" field when that field will always contain a 16? I think you could easily get away with half the size here (just 4 bytes tag and 4 bytes data size). (While we're at it, the whole table header is also a bit large... do we really expect to need 2^32 possible versions? You could easily shrink that down to 16 bytes.)
3. It's not exactly clear what the table header's "version" field is supposed to mean. What exactly would cause us to increase the version? What should a reader do when it reads an unknown version field? It would probably be a good idea for the final document to specify that in more detail, and I'd maybe consider splitting the version number in a major version and minor version byte (major version = fully incompatible to older readers; minor version = added meaning to previously reserved fields, but older readers can continue to read the fields they know about).
4. I don't think a strict definition of entry type ranges is a good idea, particularly if "standard" and "OEM" are the only two options. In my experience such systems (e.g. same situation with ARM SMC FIDs) just mean that nothing ever gets added in the "standard" bucket because the barrier to inclusion is way too high, and so in practice everyone will throw every use case they have into the remaining free-for-all bucket ("OEM" in this case), which quickly leads to a mess of overlapping IDs and in the end nobody can ever share a tag between two different platforms even if they actually end up needing the same thing. I think you can't expect things like this to self-organize in advance, and the only way it can work in practice is that things which were originally added by one contributor become a de-facto standard over time as others adopt them. The way to allow and promote that would be to have a simple, low-barrier registration mechanism for new tags -- e.g. just by submitting a patch to the repository that also holds the spec itself which would just need the tag name, ID, and simple specification of the contained data, open for everyone and every use case they might have. With 4 billion tags, I think we wouldn't need to worry about running out any time soon. This would allow all the different firmware projects and hardware vendors using this standard to iterate quickly on their own needs, deprecate tags and add new versions as needed, and if over time specific tags become widely used/useful de-facto standards other stakeholders can start supporting them without having to worry about tag clashes. (If you want more organization than just one big free-for-all bucket you could still have that, of course... e.g. allocate smaller chunks of 1K tags each to each firmware project or hardware vendor or whatever as needed. But the important part is that the barrier to adding new tags should be low and the chunks should not overlap, so the 1K tags allocated to U-Boot should be different from the 1K tags allocated to TF-A, and if U-Boot comes up with a good tag definition where we end up seeing that it would be useful if TF-A reused the same format, we can just use that tag without risk of it already being defined as something else in TF-A context.)
5. Embedding large data structures like a full FDT or ACPI table directly by value in this structure seems like a bad idea and too inflexible in practice. These kinds of structures may be generated at different points in the boot process (some of which may or may not use the transfer list standard), they may need to be modified (including growing/shrinking in size) at various points, and they may need to be placed in special locations in memory. Besides, the transfer list design requires it to be copied in various cases (e.g. to expand beyond max_length), so it seems primarily suited for small blocks of data and including a huge external structure by value would be very inefficient. I'd recommend making reference tags for these instead that just include a (physical, absolute) pointer to the actual structure, its length, and maybe a checksum.
Hi Julius,
Thanks for reviewing the draft proposal, and for your comments.
Last year’s discussion concluded with the agreement that, as a next step, we would draft a proposal of the data structure [1].
That draft is the DEN0135 document [2]. I realise that I haven’t made it sufficiently explicit in this thread that the DEN0135 document [2] is still at draft quality (see ALP maturity called out in the title page and footers). Please do not consider this a finished document 😊. We’ve been iterating on this pdf as we gather feedback from the open-source communities. It's becoming clear that we need to move this debate to a more suitable forum. That’s why we’re proposing, in this thread, to move the document to trustedfirmware.org.
There (and addressing the bottom half of your 4th point), the intent is to take new tag proposals as patch submissions. As you point out, this creates a low-barrier for new tags to be requested, suiting the needs of the open source communities that do adopt this data structure. First, we need to gather consensus on the initial document release. We want to use the repo in tf.org as a medium to mature the data structure definition and track the feedback/discussions.
You raise very good points. Some do clash with feedback we've received so far. That further motivates the need to host this document in a repo. Once that's done, we'll have a suitable forum to discuss these distinct perspectives.
- Where did the requirement for 16-byte alignment come from?
The requirement originated in feedback from the u-boot community, please see [3]. The argument for 16-byte alignment is that some data-structures have to be 16-byte aligned. We ought to meet that requirement if we ever want to carry those structures.
It seems a bit excessive considering that no data field is actually larger than 4 bytes (and it looks like it influenced several other choices in the spec, e.g. unnecessarily large transfer entry header because the bytes to the alignment boundary have to be filled anyway).
None of the current standard entries contain a field larger than 4-bytes. It's plausible that in the future there may be standard entries with a 64-bit field. Hence, I believe we need entries to be at least 8-byte aligned.
- The table entry header seems unnecessarily large. What's the point of
including a "header length" field when that field will always contain a 16?
The "header length" enables the table entry header to be extended if we ever need that in the future. Changes to the TE header are backwards compatible. A consumer obtains the address of the data by adding the "header length" to the entry's base address.
While we're at it, the whole table header is also a bit large... do we really expect to need 2^32 possible versions? You could easily shrink that down to 16 bytes.)
- It's not exactly clear what the table header's "version" field is supposed to
mean. What exactly would cause us to increase the version?
The TL header version is incremented when we do a new release of the document. At a new document release, we can introduce one or more new standard tags and also append fields to the TL header.
What should a reader do when it reads an unknown version field? It would probably be a good idea for the final document to specify that in more detail, and I'd maybe consider splitting the version number in a major version and minor version byte (major version = fully incompatible to older readers; minor version = added meaning to previously reserved fields, but older readers can continue to read the fields they know about).
Agreed. I see this point requires further discussion and agreement between all involved.
- I don't think a strict definition of entry type ranges is a good idea,
particularly if "standard" and "OEM" are the only two options.
As mentioned above, the standard tag range ought to have a low-barrier for new tags to be requested. The OEM range allows FW integrators to carry platform-specific entries that may not be generalizable.
- Embedding large data structures like a full FDT or ACPI table directly by value
in this structure seems like a bad idea and too inflexible in practice.
Note that the FDT can still be pointed to by X0/R2 without having to be included by value in the TL. The feedback we received so far is: data should be included in the TL as that simplifies procedures, such as relocating the data, without having to worry about stale pointers. Having said that, maybe we can debate this point and find a compromise that works for all!
Regards, Jose
[1] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.o... [2] https://developer.arm.com/documentation/den0135/a [3] https://www.mail-archive.com/u-boot@lists.denx.de/msg441605.html
That draft is the DEN0135 document [2]. I realise that I haven’t made it sufficiently explicit in this thread that the DEN0135 document [2] is still at draft quality (see ALP maturity called out in the title page and footers). Please do not consider this a finished document 😊. We’ve been iterating on this pdf as we gather feedback from the open-source communities. It's becoming clear that we need to move this debate to a more suitable forum. That’s why we’re proposing, in this thread, to move the document to trustedfirmware.org.
Understood. FWIW, I totally support hosting this on TF.org, I think that makes sense considering that it has good co-development infrastructure that is open to everyone (e.g. Gerrit) already set up and that it's easy to administer for you guys who are currently organizing this. I don't really see Simon's concern regarding "captive to one project" -- this whole spec is an attempt to cooperate better, not work against each other, so there shouldn't need to be fights about who "owns" it (just pick a place with good infrastructure and let everyone access it there). If perhaps the real concern is rather related to who controls future spec updates, tag assignments, etc. then I think we should address those directly and define clear processes that we can all feel comfortable with, rather than implying that the final word on these is left to whoever controls the hosting platform.
The requirement originated in feedback from the u-boot community, please see [3]. The argument for 16-byte alignment is that some data-structures have to be 16-byte aligned. We ought to meet that requirement if we ever want to carry those structures.
I still really think that this is wasting space to the point where it defeats the purpose of this being a "lightweight" data handoff structure. Consider that all of the bl_aux_params tags in TF-A (which I described as one of the existing mechanisms for this kind of thing in last year's thread) are 8 bytes or less, which means that for these use cases the transfer list would have over 3 times as much overhead as actual data. There seems to be some disconnect here since the stated idea for this is to be a lightweight handoff mechanism to pass individual pieces of information directly, yet all the initially proposed tags are just for wrapping large existing hand off blobs that bundle lot of information in their own structures (giving it a very xkcd.com/927-like feel), and the spec design seems to be tuned toward the latter use cases.
If individual tags have special alignment requirements, they could just deal with that internally (by including another offset field in the "data" part) without dragging every other tag in the structure down with them. Alternatively, if you really want to keep this I think an acceptable compromise could be to keep the 16-byte entry alignment but reduce the transfer entry header size to 8 bytes, and then allow tags that don't require 16 byte data alignment to immediately use the following 8 bytes for data. That way, entries with 8 bytes or less of data could fit both header and data into a single 16 byte block which drastically cuts down on overhead for small data.
- The table entry header seems unnecessarily large. What's the point of
including a "header length" field when that field will always contain a 16?
The "header length" enables the table entry header to be extended if we ever need that in the future. Changes to the TE header are backwards compatible. A consumer obtains the address of the data by adding the "header length" to the entry's base address.
I really don't see this as likely enough to be worth paying that big upfront cost of making the whole structure so much less efficient to start with. I think 8 bytes are already the most you'd reasonably want to pay for header overhead, I find it highly improbable that we'd want to introduce more fields later to make that even bigger. If there's desire for optional large annotations (e.g. per-entry cryptographic hashes) in the future, I'd rather suggest implementing that as "companion entries" with special tags that are written right behind the entry they refer to, rather than try squeezing it all into a single header structure. If you want to leave some room for simple bit flags, I would suggest masking out the high byte of the data length field instead (i.e. splitting it into 3 and 1) -- it doesn't seem like anyone would want to pass a data item larger than 32 MB in a firmware information structure anyway.
The TL header version is incremented when we do a new release of the document. At a new document release, we can introduce one or more new standard tags and also append fields to the TL header.
FWIW I would suggest decoupling the data structure version from the document version and only increasing the data structure version when the actual structure changes in a meaningful way. Documents may be amended to improve clarity or add more detailed explanations that don't always require parsers to actually act differently, so increasing version numbers in the code for those cases would seem to just unnecessarily confuse things.
- I don't think a strict definition of entry type ranges is a good idea,
particularly if "standard" and "OEM" are the only two options.
As mentioned above, the standard tag range ought to have a low-barrier for new tags to be requested. The OEM range allows FW integrators to carry platform-specific entries that may not be generalizable.
Okay. This definitely needs to be specified in more detail and give more guidance about where tags should be allocated for what case, because my first impression from reading the doc was apparently quite different than yours. I think the spec should strongly encourage allocating a global tag whenever there might be the slightest possibility that other platforms might want to use it later. For example, maybe we can define the guideline that anything referred to by any code checked into the upstream of any open source project (U-Boot, TF-A, whatever) should always be a global tag. I guess a small range of OEM tags makes sense for closed source downstream code that isn't shared with anyone, but that should be the only use case for that (so that open source projects can always freely collaborate and adopt each others' standards without the risk of tag clashes).
The feedback we received so far is: data should be included in the TL as that simplifies procedures, such as relocating the data, without having to worry about stale pointers.
I generally agree with that sentiment, but I also don't see including a whole FDT or ACPI table in this structure as a normal case -- that's more of a pathological edge case necessary to interface with environments that use these other hand-off structures (because all these are really just different ways to do the same thing, and there's no real benefit from wrapping one in the other... it's just an unfortunate necessity due to different boot stages expecting different things). I really expect these to be mostly independent (and come with their own placement requirements, which is a problem for direct inclusion), I would expect at least some platforms to need the flexibility of keeping them separate and therefore think it would be better to standardize on that. But if other platforms really think it's important to have these inline in the transfer list for their use case, you could offer separate tags for both.
Hi Julius,
Thank you for the comments!
As mentioned yesterday, the FW Handoff document is now publicly accessible from [1] -- note that this does not constitute a full document release. Hopefully we'll be able to use that repo to agree on the open items before we do an initial full release of the document.
If perhaps the real concern is rather related to who controls future spec updates, tag assignments, etc. then I think we should address those directly and define clear processes that we can all feel comfortable with, rather than implying that the final word on these is left to whoever controls the hosting platform.
The proposal is that the tag assignments are handled via a PR to [1]. A PR should provide reasoning for the proposed entry layout as well as a description of the use-case being serviced. The community is invited to comment on the pull requests. We should aim to have community consensus before merging any PR.
I really don't see this as likely enough to be worth paying that big upfront cost of making the whole structure so much less efficient to start with. I think 8 bytes are already the most you'd reasonably want to pay for header overhead, I find it highly improbable that we'd want to introduce more fields later to make that even bigger. If there's desire for optional large annotations (e.g. per-entry cryptographic hashes) in the future, I'd rather suggest implementing that as "companion entries" with special tags that are written right behind the entry they refer to, rather than try squeezing it all into a single header structure. If you want to leave some room for simple bit flags, I would suggest masking out the high byte of the data length field instead (i.e. splitting it into 3 and 1) -- it doesn't seem like anyone would want to pass a data item larger than 32 MB in a firmware information structure anyway.
It seems sensible that data contained in an entry should not exceed 32MB. Could we still accommodate the notion of a hdr_size in the TL entry header? We'd have the following fields on an 8-byte TE header: tag_id -- 4 bytes hdr_size -- 1 byte Data_size -- 3 bytes
If this proposal addresses your concerns, we should create a PR for it to be discussed!
FWIW I would suggest decoupling the data structure version from the document version and only increasing the data structure version when the actual structure changes in a meaningful way. Documents may be amended to improve clarity or add more detailed explanations that don't always require parsers to actually act differently, so increasing version numbers in the code for those cases would seem to just unnecessarily confuse things.
The document and the TL version are now separate. The document version is defined [2]. The TL version is present in the header [3] and shall be incremented when changes are introduced to either the TL header or the entries which a consumer must discover (via the data TL version).
I think the spec should strongly encourage allocating a global tag whenever there might be the slightest possibility that other platforms might want to use it later
The document states that any tag in the standard range must be allocated in the spec before being used in code [4] -- what this is really saying is: until a tag id is allocated anyone can request it. So code written assuming a particular standard tag id (absent from the spec) may have to change later if someone in the meantime allocates that tag id for a different entry type. I believe it’s a per-community policy to prevent code submissions containing standard tag ids until these have been reserved in this spec. This is the advised policy, but this spec cannot mandate it, rather we must rely on the communities to enforce this.
I would expect at least some platforms to need the flexibility of keeping them separate and therefore think it would be better to standardize on that. But if other platforms really think it's important to have these inline in the transfer list for their use case, you could offer separate tags for both.
For the current set of entries, the data is included within the TL. Note that this does not prevent newer entries from pointing to off-TL data. Could this be handled on a case-by-case basis, via pull request, after an initial full release of the spec?
Regards, Jose
[1] https://github.com/FirmwareHandoff/firmware_handoff [2]https://github.com/FirmwareHandoff/firmware_handoff/blob/1ce91def658a4fe26b9... [3] https://github.com/FirmwareHandoff/firmware_handoff/blob/main/source/transfe... [4]https://github.com/FirmwareHandoff/firmware_handoff/blame/main/source/transf...
Hi Jose,
Apologies for the late response, I had to find some time to dig back into this topic first.
The proposal is that the tag assignments are handled via a PR to [1]. A PR should provide reasoning for the proposed entry layout as well as a description of the use-case being serviced. The community is invited to comment on the pull requests. We should aim to have community consensus before merging any PR.
I think this is a good approach if we understand "community consensus" to be a quick, couple-of-days review cycle like we have for TF-A patches that doesn't create a huge threshold for additions, and it's fine for entries to be very specific to the needs of a single platform when necessary. But I think the document should still have more guidance about when standard vs. non-standard tags should be used, and in particular just strongly discourage the use of non-standard tags in general (basically I think they should only be used for experiments that won't reach production code, or for cases where all code reading or writing that tag is closed source). I think the spec should work really hard in its language and guidance to discourage a situation where firmware components just assign a non-standard tag to something because it's easier and avoids the hassle of submitting something you don't think you'd want to share at the time, and then later it becomes entrenched and reused and eventually everyone has to pay attention to not reuse the non-standard tags that accidentally acquired a fixed meaning in enough implementations to become a problem. (It may also make sense to just make the standard tag range much bigger than the non-standard range, to make it clearer that the majority of things are expected to go in there.)
It seems sensible that data contained in an entry should not exceed 32MB. Could we still accommodate the notion of a hdr_size in the TL entry header? We'd have the following fields on an 8-byte TE header: tag_id -- 4 bytes hdr_size -- 1 byte Data_size -- 3 bytes
Yeah, that seems like a reasonable compromise, if you really think that variable header length is useful.
I still think the table header could be smaller as well (down to 16 bytes from 32 by just avoiding so much reserved space).
Do you want me to send those suggestions as pull requests to make them easier to discuss?
The document states that any tag in the standard range must be allocated in the spec before being used in code [4] -- what this is really saying is: until a tag id is allocated anyone can request it. So code written assuming a particular standard tag id (absent from the spec) may have to change later if someone in the meantime allocates that tag id for a different entry type. I believe it’s a per-community policy to prevent code submissions containing standard tag ids until these have been reserved in this spec. This is the advised policy, but this spec cannot mandate it, rather we must rely on the communities to enforce this.
I would say the spec should mandate that, honestly. If there is a standardized range, no firmware implementing transfer lists according to this spec should use tags from that range until they have been defined in the global repository. On the flip side, we need to make the process of defining new tags fast and painless so that this requirement doesn't become a burden.
For the current set of entries, the data is included within the TL. Note that this does not prevent newer entries from pointing to off-TL data. Could this be handled on a case-by-case basis, via pull request, after an initial full release of the spec?
Yes, we can leave it like this and later add tags for the off-list equivalent of the tags defined there if necessary.
Hi,
On Wed, 30 Nov 2022 at 14:52, Julius Werner jwerner@chromium.org wrote:
Hi Jose,
Apologies for the late response, I had to find some time to dig back into this topic first.
The proposal is that the tag assignments are handled via a PR to [1]. A PR should provide reasoning for the proposed entry layout as well as a description of the use-case being serviced. The community is invited to comment on the pull requests. We should aim to have community consensus before merging any PR.
I think this is a good approach if we understand "community consensus" to be a quick, couple-of-days review cycle like we have for TF-A patches that doesn't create a huge threshold for additions, and it's fine for entries to be very specific to the needs of a single platform when necessary. But I think the document should still have more guidance about when standard vs. non-standard tags should be used, and in particular just strongly discourage the use of non-standard tags in general (basically I think they should only be used for experiments that won't reach production code, or for cases where all code reading or writing that tag is closed source). I think the spec should work
That's right
really hard in its language and guidance to discourage a situation where firmware components just assign a non-standard tag to something because it's easier and avoids the hassle of submitting something you don't think you'd want to share at the time, and then later it becomes entrenched and reused and eventually everyone has to pay attention to not reuse the non-standard tags that accidentally acquired a fixed meaning in enough implementations to become a problem. (It may also make sense to just make the standard tag range much bigger than the non-standard range, to make it clearer that the majority of things are expected to go in there.)
One way to enforce that is for firmware components to refuse to accept a transfer list with unknown tags, assuming that all firmware is built from source at the same time. It might be a pain, but I suspect it could help avoid security problems, where bad actors use the structure to pass code to a firmware blob, etc.
It seems sensible that data contained in an entry should not exceed 32MB. Could we still accommodate the notion of a hdr_size in the TL entry header? We'd have the following fields on an 8-byte TE header: tag_id -- 4 bytes hdr_size -- 1 byte Data_size -- 3 bytes
Yeah, that seems like a reasonable compromise, if you really think that variable header length is useful.
I still think the table header could be smaller as well (down to 16 bytes from 32 by just avoiding so much reserved space).
Do you want me to send those suggestions as pull requests to make them easier to discuss?
I have to disagree here. The context is gone in this email, so I cannot reply to the comments there. But I think it was talking about using the transfer list for very tiny structures, just a few words. That is not the intent. Firstly we should really have TF-A use the device tree (embedded in the transfer list) instead of replicating bl_params in this structure. Secondly, we should group things into a C structure that is at least 16-32 bytes long.
So I don't see a need for a smaller header. The 16-byte alignment works well with ACPI tables and we should not be using this structure for tiny things where the 8-byte overhead is significant. Believe me I have spent plenty of time dealing with memory space in U-Boot TPL, but
As to the maximum size, I can see how things might grow larger, for example to pass large amounts of data between firmware pieces, such as images, executables, even a ramdisk. It seems short-sighted to limit the size to 32MB.
For the header size, this provides for backwards compatibility in the case where we need to store more there. I don't think this is a big deal and certainly one byte should be enough...
But overall I think having a 16-byte header makes sense.
The document states that any tag in the standard range must be allocated in the spec before being used in code [4] -- what this is really saying is: until a tag id is allocated anyone can request it. So code written assuming a particular standard tag id (absent from the spec) may have to change later if someone in the meantime allocates that tag id for a different entry type. I believe it’s a per-community policy to prevent code submissions containing standard tag ids until these have been reserved in this spec. This is the advised policy, but this spec cannot mandate it, rather we must rely on the communities to enforce this.
I would say the spec should mandate that, honestly. If there is a standardized range, no firmware implementing transfer lists according to this spec should use tags from that range until they have been defined in the global repository. On the flip side, we need to make the process of defining new tags fast and painless so that this requirement doesn't become a burden.
Agreed on both points.
For the current set of entries, the data is included within the TL. Note that this does not prevent newer entries from pointing to off-TL data. Could this be handled on a case-by-case basis, via pull request, after an initial full release of the spec?
Yes, we can leave it like this and later add tags for the off-list equivalent of the tags defined there if necessary.
Ideally we should avoid this, since it makes it hard to relocate things. Anyway I agree we can add it later if really needed.
Regards, Simon
Okay, FWIW I created a pull request with my suggestions here: https://github.com/FirmwareHandoff/firmware_handoff/pull/4
That should make it easier to discuss specific details, hopefully. As I was looking at the header size and format more closely I noticed that the checksum calculation and some details about what the size fields mean seems underspecified right now, so I suggested a change for that as well. Let me know what you think.
One way to enforce that is for firmware components to refuse to accept a transfer list with unknown tags, assuming that all firmware is built from source at the same time. It might be a pain, but I suspect it could help avoid security problems, where bad actors use the structure to pass code to a firmware blob, etc.
I don't think that's feasible, particularly if tag IDs are not allocated in order (which I think is useful so that we can try to group similar tags by vendor, e.g. if say MediaTek wanted to add some tags they could just find some nice round number in the tag space, say 0x200, and then start allocating 0x201, 0x202, 0x203, etc. while the next vendor can do their own thing at 0x300 instead). It also creates automatic incompatibilities if you put newer and older components in the same image, which I think is something we explicitly don't want. I think the right policy to encourage compatibility is that unknown tags just get ignored.
I have to disagree here. The context is gone in this email, so I cannot reply to the comments there. But I think it was talking about using the transfer list for very tiny structures, just a few words. That is not the intent. Firstly we should really have TF-A use the device tree (embedded in the transfer list) instead of replicating bl_params in this structure. Secondly, we should group things into a C structure that is at least 16-32 bytes long.
Uhh... okay, sorry, but then you're going against everything that was discussed in this thread and in the previous discussions last year. The whole idea of making this a tagged list came from the argument that other structures (like FDT) are unnecessarily bulky and complex for some of the intended use cases (like replacing TF-A bl_aux_params). I mean, what's the point of even having a transfer list if all it does is wrap an FDT? You could just use the FDT directly! I think all the 4 tags that are currently specified in that document are pathological edge cases that may be necessary for legacy applications but represent the opposite of how this structure *should* be used in the ideal case. We don't need just more wrappers for the stuff that we already have, that doesn't promote interoperability in any way. If project A only wants to use HOBs but not FDT, and project B only wants to use FDT but not HOBs, then creating a transfer list structure that can wrap both an FDT and a HOB in a TE is pointless and not going to fix anything (project A will still not be able to parse the FDT-wrapping TE, and project B will still not be able to parse the HOB-wrapping TE). The only way this whole exercise can actually be useful and improve anything is if we can use it to eventually (slowly) get away from both HOBs and FDTs towards a point where all data is encoded in TEs directly, without any extra layers.
So I don't see a need for a smaller header. The 16-byte alignment works well with ACPI tables and we should not be using this structure for tiny things where the 8-byte overhead is significant.
FWIW, the compromise proposal now preserves the 16-byte TE alignment. (8 byte header, then small entries can use the next 8 bytes for data and the whole TE is only 16 bytes with only up to 7 bytes of forced padding. Once you need more than 8 bytes of data it gets a bit more ugly but at that point I guess you're not really a super small entry anymore.)
Hi,
On Wed, 30 Nov 2022 at 16:48, Julius Werner jwerner@chromium.org wrote:
Okay, FWIW I created a pull request with my suggestions here: https://github.com/FirmwareHandoff/firmware_handoff/pull/4
That should make it easier to discuss specific details, hopefully. As I was looking at the header size and format more closely I noticed that the checksum calculation and some details about what the size fields mean seems underspecified right now, so I suggested a change for that as well. Let me know what you think.
One way to enforce that is for firmware components to refuse to accept a transfer list with unknown tags, assuming that all firmware is built from source at the same time. It might be a pain, but I suspect it could help avoid security problems, where bad actors use the structure to pass code to a firmware blob, etc.
I don't think that's feasible, particularly if tag IDs are not allocated in order (which I think is useful so that we can try to group similar tags by vendor, e.g. if say MediaTek wanted to add some tags they could just find some nice round number in the tag space, say 0x200, and then start allocating 0x201, 0x202, 0x203, etc. while the next vendor can do their own thing at 0x300 instead). It also creates automatic incompatibilities if you put newer and older components in the same image, which I think is something we explicitly don't want. I think the right policy to encourage compatibility is that unknown tags just get ignored.
I have to disagree here. The context is gone in this email, so I cannot reply to the comments there. But I think it was talking about using the transfer list for very tiny structures, just a few words. That is not the intent. Firstly we should really have TF-A use the device tree (embedded in the transfer list) instead of replicating bl_params in this structure. Secondly, we should group things into a C structure that is at least 16-32 bytes long.
Uhh... okay, sorry, but then you're going against everything that was discussed in this thread and in the previous discussions last year. The whole idea of making this a tagged list came from the argument that other structures (like FDT) are unnecessarily bulky and complex for some of the intended use cases (like replacing TF-A bl_aux_params). I mean, what's the point of even having a transfer list if all it does is wrap an FDT? You could just use the FDT directly! I think all the 4 tags that are currently specified in that document are pathological edge cases that may be necessary for legacy applications but represent the opposite of how this structure *should* be used in the ideal case. We don't need just more wrappers for the stuff that we already have, that doesn't promote interoperability in any way. If project A only wants to use HOBs but not FDT, and project B only wants to use FDT but not HOBs, then creating a transfer list structure that can wrap both an FDT and a HOB in a TE is pointless and not going to fix anything (project A will still not be able to parse the FDT-wrapping TE, and project B will still not be able to parse the HOB-wrapping TE). The only way this whole exercise can actually be useful and improve anything is if we can use it to eventually (slowly) get away from both HOBs and FDTs towards a point where all data is encoded in TEs directly, without any extra layers.
I don't know where that idea came from and I thought I read the same threads. I think FDT is the best data structure in general, because it is self-describing, can be understood just by dumping it and we can use schema tools to validate things, and many other reasons. Overall, as an industry, I believe it is best to get away from ad-hoc binary structures.
So all projects which need to send or receive complex data should use FDT.
For replacing bl_aux_params, I think FDT makes sense in general, but there may be situations where a few TEs are useful though.
The point of the transfer list is to deal with cases where we don't have enough memory / CPU power to use libfdt. This happens very early in boot, and in that case it is helpful to pass small data structures containing useful information.
Actually even just grouping data structures (FDT, ACPI things) in a TL is useful, since we have a unified way to find them all.
I replied on the github PR also.
So I don't see a need for a smaller header. The 16-byte alignment works well with ACPI tables and we should not be using this structure for tiny things where the 8-byte overhead is significant.
FWIW, the compromise proposal now preserves the 16-byte TE alignment. (8 byte header, then small entries can use the next 8 bytes for data and the whole TE is only 16 bytes with only up to 7 bytes of forced padding. Once you need more than 8 bytes of data it gets a bit more ugly but at that point I guess you're not really a super small entry anymore.)
I initially thought that was fine, but then I realised that this loses the 16-byte alignment of the entries.
Regards, SImon
Hi
Just a few comments:
* I agree with Julius that in general it isn't feasible for firmware to refuse to accept a TL with unknown tags. The structures are defined to help with backwards/forwards compatibility. To assume that all firmware is built at the same time is a special case. A particular firmware implementation could have such a policy, but I don't think we can specify this. I like the idea of defining a tag allocation policy but I'm not sure it's a good idea for this to be in the main specification as this could rapidly change depending on how the spec is used in practice. I've commented on the PR to this effect.
* I don't think we’re all going to agree on how we're going to use the TL, which is part of the reason for its existence. At least one of the use-cases is combining the passing of standard structures like FDT/HOBs through to the normal world firmware and/or OS, with the passing of smaller structures between more tightly integrated firmware components. In a system where BL31 (for example) has to parse FDT anyway, it might make sense to put all data in FDT format, but in other systems, BL31 might just relay the FDT, so adding an FDT parser might add unnecessary overhead. I don't think we need to mandate usage for now - patterns will emerge.
* Using TEs throughout is probably only going to work for vertically integrated systems, but that is clearly also an important case, so optimizing the structures for size in those cases is reasonable. I've also commented on the PR about this.
Cheers
Dan.
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 02 December 2022 17:54 To: Julius Werner jwerner@chromium.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Rob Herring Rob.Herring@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Sivasakthivel Nainar snainar@amperecomputing.com; Samer El-Haj- Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: Re: [TF-A] [RFC] Proposed location to host the firmware handoff specification.
Hi,
On Wed, 30 Nov 2022 at 16:48, Julius Werner jwerner@chromium.org wrote:
Okay, FWIW I created a pull request with my suggestions here: https://github.com/FirmwareHandoff/firmware_handoff/pull/4
That should make it easier to discuss specific details, hopefully. As I was looking at the header size and format more closely I noticed that the checksum calculation and some details about what the size fields mean seems underspecified right now, so I suggested a change for that as well. Let me know what you think.
One way to enforce that is for firmware components to refuse to accept a transfer list with unknown tags, assuming that all firmware is built from source at the same time. It might be a pain, but I suspect it could help avoid security problems, where bad actors use the structure to pass code to a firmware blob, etc.
I don't think that's feasible, particularly if tag IDs are not allocated in order (which I think is useful so that we can try to group similar tags by vendor, e.g. if say MediaTek wanted to add some tags they could just find some nice round number in the tag space, say 0x200, and then start allocating 0x201, 0x202, 0x203, etc. while the next vendor can do their own thing at 0x300 instead). It also creates automatic incompatibilities if you put newer and older components in the same image, which I think is something we explicitly don't want. I think the right policy to encourage compatibility is that unknown tags just get ignored.
I have to disagree here. The context is gone in this email, so I cannot reply to the comments there. But I think it was talking about using the transfer list for very tiny structures, just a few words. That is not the intent. Firstly we should really have TF-A use the device tree (embedded in the transfer list) instead of replicating bl_params in this structure. Secondly, we should group things into a C structure that is at least 16-32 bytes long.
Uhh... okay, sorry, but then you're going against everything that was discussed in this thread and in the previous discussions last year. The whole idea of making this a tagged list came from the argument that other structures (like FDT) are unnecessarily bulky and complex for some of the intended use cases (like replacing TF-A bl_aux_params). I mean, what's the point of even having a transfer list if all it does is wrap an FDT? You could just use the FDT directly! I think all the 4 tags that are currently specified in that document are pathological edge cases that may be necessary for legacy applications but represent the opposite of how this structure *should* be used in the ideal case. We don't need just more wrappers for the stuff that we already have, that doesn't promote interoperability in any way. If project A only wants to use HOBs but not FDT, and project B only wants to use FDT but not HOBs, then creating a transfer list structure that can wrap both an FDT and a HOB in a TE is pointless and not going to fix anything (project A will still not be able to parse the FDT-wrapping TE, and project B will still not be able to parse the HOB-wrapping TE). The only way this whole exercise can actually be useful and improve anything is if we can use it to eventually (slowly) get away from both HOBs and FDTs towards a point where all data is encoded in TEs directly, without any extra layers.
I don't know where that idea came from and I thought I read the same threads. I think FDT is the best data structure in general, because it is self- describing, can be understood just by dumping it and we can use schema tools to validate things, and many other reasons. Overall, as an industry, I believe it is best to get away from ad-hoc binary structures.
So all projects which need to send or receive complex data should use FDT.
For replacing bl_aux_params, I think FDT makes sense in general, but there may be situations where a few TEs are useful though.
The point of the transfer list is to deal with cases where we don't have enough memory / CPU power to use libfdt. This happens very early in boot, and in that case it is helpful to pass small data structures containing useful information.
Actually even just grouping data structures (FDT, ACPI things) in a TL is useful, since we have a unified way to find them all.
I replied on the github PR also.
So I don't see a need for a smaller header. The 16-byte alignment works well with ACPI tables and we should not be using this structure for tiny things where the 8-byte overhead is significant.
FWIW, the compromise proposal now preserves the 16-byte TE alignment. (8 byte header, then small entries can use the next 8 bytes for data and the whole TE is only 16 bytes with only up to 7 bytes of forced padding. Once you need more than 8 bytes of data it gets a bit more ugly but at that point I guess you're not really a super small entry anymore.)
I initially thought that was fine, but then I realised that this loses the 16-byte alignment of the entries.
Regards, SImon
Hi Dan, Julius,
On Tue, 6 Dec 2022 at 06:51, Dan Handley Dan.Handley@arm.com wrote:
Hi
Just a few comments:
- I agree with Julius that in general it isn't feasible for firmware to refuse to accept a TL with unknown tags. The structures are defined to help with backwards/forwards compatibility. To assume that all firmware is built at the same time is a special case. A particular firmware implementation could have such a policy, but I don't think we can specify this. I like the idea of defining a tag allocation policy but I'm not sure it's a good idea for this to be in the main specification as this could rapidly change depending on how the spec is used in practice. I've commented on the PR to this effect.
That all sounds reasonable to me.
- I don't think we’re all going to agree on how we're going to use the TL, which is part of the reason for its existence. At least one of the use-cases is combining the passing of standard structures like FDT/HOBs through to the normal world firmware and/or OS, with the passing of smaller structures between more tightly integrated firmware components. In a system where BL31 (for example) has to parse FDT anyway, it might make sense to put all data in FDT format, but in other systems, BL31 might just relay the FDT, so adding an FDT parser might add unnecessary overhead. I don't think we need to mandate usage for now - patterns will emerge.
The main concern I have here is that TL not be used for non-trivial binary formats (say more than a C struct or table of them). We have so many of those sorts of things. The overhead of FDT is certainly a concern, but some of these components have hundreds of KB of code anyway. Adding another 5KB for all the convenience and visibility shouldn't matter.
- Using TEs throughout is probably only going to work for vertically integrated systems, but that is clearly also an important case, so optimizing the structures for size in those cases is reasonable. I've also commented on the PR about this.
OK.
- Simon
Cheers
Dan.
-----Original Message----- From: Simon Glass sjg@chromium.org Sent: 02 December 2022 17:54 To: Julius Werner jwerner@chromium.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Dan Handley Dan.Handley@arm.com; Rob Herring Rob.Herring@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Sivasakthivel Nainar snainar@amperecomputing.com; Samer El-Haj- Mahmoud Samer.El-Haj-Mahmoud@arm.com; nd nd@arm.com Subject: Re: [TF-A] [RFC] Proposed location to host the firmware handoff specification.
Hi,
On Wed, 30 Nov 2022 at 16:48, Julius Werner jwerner@chromium.org wrote:
Okay, FWIW I created a pull request with my suggestions here: https://github.com/FirmwareHandoff/firmware_handoff/pull/4
That should make it easier to discuss specific details, hopefully. As I was looking at the header size and format more closely I noticed that the checksum calculation and some details about what the size fields mean seems underspecified right now, so I suggested a change for that as well. Let me know what you think.
One way to enforce that is for firmware components to refuse to accept a transfer list with unknown tags, assuming that all firmware is built from source at the same time. It might be a pain, but I suspect it could help avoid security problems, where bad actors use the structure to pass code to a firmware blob, etc.
I don't think that's feasible, particularly if tag IDs are not allocated in order (which I think is useful so that we can try to group similar tags by vendor, e.g. if say MediaTek wanted to add some tags they could just find some nice round number in the tag space, say 0x200, and then start allocating 0x201, 0x202, 0x203, etc. while the next vendor can do their own thing at 0x300 instead). It also creates automatic incompatibilities if you put newer and older components in the same image, which I think is something we explicitly don't want. I think the right policy to encourage compatibility is that unknown tags just get ignored.
I have to disagree here. The context is gone in this email, so I cannot reply to the comments there. But I think it was talking about using the transfer list for very tiny structures, just a few words. That is not the intent. Firstly we should really have TF-A use the device tree (embedded in the transfer list) instead of replicating bl_params in this structure. Secondly, we should group things into a C structure that is at least 16-32 bytes long.
Uhh... okay, sorry, but then you're going against everything that was discussed in this thread and in the previous discussions last year. The whole idea of making this a tagged list came from the argument that other structures (like FDT) are unnecessarily bulky and complex for some of the intended use cases (like replacing TF-A bl_aux_params). I mean, what's the point of even having a transfer list if all it does is wrap an FDT? You could just use the FDT directly! I think all the 4 tags that are currently specified in that document are pathological edge cases that may be necessary for legacy applications but represent the opposite of how this structure *should* be used in the ideal case. We don't need just more wrappers for the stuff that we already have, that doesn't promote interoperability in any way. If project A only wants to use HOBs but not FDT, and project B only wants to use FDT but not HOBs, then creating a transfer list structure that can wrap both an FDT and a HOB in a TE is pointless and not going to fix anything (project A will still not be able to parse the FDT-wrapping TE, and project B will still not be able to parse the HOB-wrapping TE). The only way this whole exercise can actually be useful and improve anything is if we can use it to eventually (slowly) get away from both HOBs and FDTs towards a point where all data is encoded in TEs directly, without any extra layers.
I don't know where that idea came from and I thought I read the same threads. I think FDT is the best data structure in general, because it is self- describing, can be understood just by dumping it and we can use schema tools to validate things, and many other reasons. Overall, as an industry, I believe it is best to get away from ad-hoc binary structures.
So all projects which need to send or receive complex data should use FDT.
For replacing bl_aux_params, I think FDT makes sense in general, but there may be situations where a few TEs are useful though.
The point of the transfer list is to deal with cases where we don't have enough memory / CPU power to use libfdt. This happens very early in boot, and in that case it is helpful to pass small data structures containing useful information.
Actually even just grouping data structures (FDT, ACPI things) in a TL is useful, since we have a unified way to find them all.
I replied on the github PR also.
So I don't see a need for a smaller header. The 16-byte alignment works well with ACPI tables and we should not be using this structure for tiny things where the 8-byte overhead is significant.
FWIW, the compromise proposal now preserves the 16-byte TE alignment. (8 byte header, then small entries can use the next 8 bytes for data and the whole TE is only 16 bytes with only up to 7 bytes of forced padding. Once you need more than 8 bytes of data it gets a bit more ugly but at that point I guess you're not really a super small entry anymore.)
I initially thought that was fine, but then I realised that this loses the 16-byte alignment of the entries.
Regards, SImon
It seems like some of us still have very different opinions about how this handoff structure should and shouldn't be used, which I really think need to be worked out and then codified in the spec before we can call the document final, because otherwise no firmware project can trust that it is safe to adopt this. My understanding was that this is supposed to be a universal handoff structure that's free to be used by any open source firmware project for any of its internal purposes at any stage of the boot flow as a standalone solution that doesn't force them to pull in dependencies to any other format. If that is the case then I think the spec should reflect this very clearly and should particularly not contain any language about deferring certain types of data to other handoff structures wrapped in the TL. It needs to be clear that projects will be allowed to freely register tags for their needs without the risk of suddenly getting told by the maintainers that they need to use an FDT for this or that instead.
On the other hand, if this is just supposed to be an early boot flow extension to FDTs, then that's very different from what I understood the goal to be and then I think the text of the spec should be honest about that front and center. In that case, I wouldn't expect much adoption beyond the ecosystem that's already deeply tied to FDT to begin with, though, and that would be far from "universal".
Hi Julius
-----Original Message----- From: Julius Werner jwerner@chromium.org Sent: 06 December 2022 04:18
It seems like some of us still have very different opinions about how this handoff structure should and shouldn't be used, which I really think need to be worked out and then codified in the spec before we can call the document final, because otherwise no firmware project can trust that it is safe to adopt this.
Yes, there are some very differing opinions on usage, but I think it's possible to accommodate multiple usage models at the same time. I agree we need to have maintainer consensus on how the spec will evolve and have this written down (e.g. the tag allocation policy).
My understanding was that this is supposed to be a universal handoff structure that's free to be used by any open source firmware project for any of its internal purposes at any stage of the boot flow as a standalone solution that doesn't force them to pull in dependencies to any other format.
That is a valid usage model, though not the only (universal) one.
If that is the case then I think the spec should reflect this very clearly and should particularly not contain any language about deferring certain types of data to other handoff structures wrapped in the TL. It needs to be clear that projects will be allowed to freely register tags for their needs without the risk of suddenly getting told by the maintainers that they need to use an FDT for this or that instead.
OK, this seems to be the crux of the problem. Is it possible for us say that users are free to register new TLs, while at the same time recommending the use of existing formats for complex structures (e.g. FDT, HOB)?
On the other hand, if this is just supposed to be an early boot flow extension to FDTs, then that's very different from what I understood the goal to be and then I think the text of the spec should be honest about that front and center. In that case, I wouldn't expect much adoption beyond the ecosystem that's already deeply tied to FDT to begin with, though, and that would be far from "universal".
That is another valid usage model. Some ecosystems are deeply wedded to FDT or HOB/ACPI (there may be others) and this spec is not going to change that. I don't think we're going to get something universal in the way you're hoping. But we might be able to at least enable better interoperability/reusability between fw components across different ecosystems.
Regards
Dan.
Hi,
The Firmware Handoff specification has just been released at version 0.9. The release can be found here: https://github.com/FirmwareHandoff/firmware_handoff/releases/tag/v0.9
Thanks to everyone that participated in the discussions about and on the spec development.
Once enough confidence has been obtained in the current specification, we will release a v1.0.
Note that PRs can be raised to request new entry types that are required for your platform.
Regards, Jose
-----Original Message----- From: Dan Handley Dan.Handley@arm.com Sent: Tuesday, December 6, 2022 4:15 PM To: Julius Werner jwerner@chromium.org; Simon Glass sjg@chromium.org Cc: Jose Marinho Jose.Marinho@arm.com; tf-a@lists.trustedfirmware.org; u-boot@lists.denx.de; boot-architecture@lists.linaro.org; Manish Pandey2 Manish.Pandey2@arm.com; Joanna Farley Joanna.Farley@arm.com; Ilias Apalodimas ilias.apalodimas@linaro.org; Matteo Carlini Matteo.Carlini@arm.com; Rob Herring Rob.Herring@arm.com; Harb Abdulhamid (harb@amperecomputing.com) harb@amperecomputing.com; Sivasakthivel Nainar snainar@amperecomputing.com; Samer El-Haj-Mahmoud <Samer.El-Haj- Mahmoud@arm.com>; nd nd@arm.com Subject: RE: [TF-A] [RFC] Proposed location to host the firmware handoff specification.
Hi Julius
-----Original Message----- From: Julius Werner jwerner@chromium.org Sent: 06 December 2022 04:18
It seems like some of us still have very different opinions about how this handoff structure should and shouldn't be used, which I really think need to be worked out and then codified in the spec before we can call the document final, because otherwise no firmware project can trust that it is safe to adopt this.
Yes, there are some very differing opinions on usage, but I think it's possible to accommodate multiple usage models at the same time. I agree we need to have maintainer consensus on how the spec will evolve and have this written down (e.g. the tag allocation policy).
My understanding was that this is supposed to be a universal handoff structure that's free to be used by any open source firmware project for any of its internal purposes at any stage of the boot flow as a standalone solution that doesn't force them to pull in dependencies to any other format.
That is a valid usage model, though not the only (universal) one.
If that is the case then I think the spec should reflect this very clearly and should particularly not contain any language about deferring certain types of data to other handoff structures wrapped in the TL. It needs to be clear that projects will be allowed to freely register tags for their needs without the risk of suddenly getting told by the maintainers that they need to use an FDT for this or that instead.
OK, this seems to be the crux of the problem. Is it possible for us say that users are free to register new TLs, while at the same time recommending the use of existing formats for complex structures (e.g. FDT, HOB)?
On the other hand, if this is just supposed to be an early boot flow extension to FDTs, then that's very different from what I understood the goal to be and then I think the text of the spec should be honest about that front and center. In that case, I wouldn't expect much adoption beyond the ecosystem that's already deeply tied to FDT to begin with, though, and that would be far from "universal".
That is another valid usage model. Some ecosystems are deeply wedded to FDT or HOB/ACPI (there may be others) and this spec is not going to change that. I don't think we're going to get something universal in the way you're hoping. But we might be able to at least enable better interoperability/reusability between fw components across different ecosystems.
Regards
Dan.
On 12/5/22 10:18 PM, Julius Werner via TF-A wrote:
It seems like some of us still have very different opinions about how this handoff structure should and shouldn't be used, which I really think need to be worked out and then codified in the spec before we can call the document final, because otherwise no firmware project can trust that it is safe to adopt this. My understanding was that this is supposed to be a universal handoff structure that's free to be used by any open source firmware project for any of its internal purposes at any stage of the boot flow as a standalone solution that doesn't force them to pull in dependencies to any other format. If that is the case then I think the spec should reflect this very clearly and should particularly not contain any language about deferring certain types of data to other handoff structures wrapped in the TL. It needs to be clear that projects will be allowed to freely register tags for their needs without the risk of suddenly getting told by the maintainers that they need to use an FDT for this or that instead.
I can see 3 types of use cases:
1. A tag that is intended to be used for information passed between different firmware projects. Such as from TF-A to EDK2 and u-boot.
2. A tag for _internal_ use within a firmware project. Here a firmware project should be free to do whatever they want, but they still will likely want to use tag IDs that will not conflict with other uses. I don't see the value of cluttering the firmware handoff spec with layouts internal to specific firmware projects.
3. A tag for experimental or other uses where standardization is not wanted.
We have plenty of bits, so why not define 3 ranges:
0x0 -- 0xf_ffff Standard tag id range. Any tag id in this range must first be allocated in this specification before being used. The allocation of the tag id requires the entry layout to be defined as well.
0x10_0000 -- 0x10_ffff Vendor tag id range. Any tag id in this range must first be allocated in this specification before being used. A vendor or project requests a range of tag IDs. The layout of entries in this range is not standardized.
0x11_0000 -- 0x11_ffff Non-standard range. A platform firmware integrator can create entries in this range without coordination with the firmware handoff specification. Different platforms are allowed to have tag ids in this range with distinct data formats. Entries in this range are not standardized.
0x12_0000 -- 0xffff_ffff Reserved
Thanks, Stuart
OK, this seems to be the crux of the problem. Is it possible for us say that users are free to register new TLs, while at the same time recommending the use of existing formats for complex structures (e.g. FDT, HOB)?
I'm not really sure what qualifies as a "complex structure" in this discussion? I think each individual TE should deal with a single concern, and separate concerns should be implemented via separate TEs (IIRC I tried to propose language in that direction in my PR)... so I agree that TEs shouldn't be "complex" (if this is what you mean by that), but I think the solution to that is just to divide the complexity down into a larger number of more simple TEs. I also agree that usually each TE should be representable by a simple C structure like I think Simon mentioned somewhere (although I think that should be a recommendation instead of an absolute requirement in case projects have long-established data structures that they want to continue using... the FDT and HOB list TEs aren't representable by a C structure either, after all).
On the other hand, if this is just supposed to be an early boot flow extension to FDTs, then that's very different from what I understood the goal to be and then I think the text of the spec should be honest about that front and center. In that case, I wouldn't expect much adoption beyond the ecosystem that's already deeply tied to FDT to begin with, though, and that would be far from "universal".
That is another valid usage model. Some ecosystems are deeply wedded to FDT or HOB/ACPI (there may be others) and this spec is not going to change that. I don't think we're going to get something universal in the way you're hoping. But we might be able to at least enable better interoperability/reusability between fw components across different ecosystems.
Sure, I didn't mean to say this should be disallowed, but the question is whether that is the only allowed use and the FDT is required to pass certain data, or whether adopters are free to choose how they represent their data and the FDT tag is just one of many. Like you said there really needs to be maintainer consensus about this that must be written down somewhere.
I don't believe there's going to be a lot of sharing between projects either to be honest, but ultimately that's what this whole thing was started for, right? I think in practice the most likely opportunities for sharing are going to be from small pieces that all projects need, like a TE defining which serial console to print debug output on. (That's another good reason to keep TEs simple, single-concern, and directly encoding data instead of wrapping another structure. In practice I assume that the projects that mostly rely on the wrapped HOB list or FDT will duplicate a few pieces of data into separate TEs to be able to share those with components that can't parse the larger structure.)
A tag that is intended to be used for information passed between different firmware projects. Such as from TF-A to EDK2 and u-boot.
A tag for _internal_ use within a firmware project. Here a firmware project should be free to do whatever they want, but they still will likely want to use tag IDs that will not conflict with other uses. I don't see the value of cluttering the firmware handoff spec with layouts internal to specific firmware projects.
This solves the accidental overlap concern but not the organically emerging standard concern. I guess we could say that a tag from the "vendor" range can be promoted to "standard" at a later point by adding it to the global spec (with its existing ID in the vendor range). This does hurt discoverability though, i.e. it's harder for someone who is trying to implement a new TE to realize that the problem has already been solved by another project in a way that would also work out well for them.
I still think it's important that tag layouts must be immutable and cannot change once allocated if they come out of that range, that's the basis for any interoperability and backwards-compatibility. And the best way to ensure that is to have them all listed in a global location. Otherwise, even if you write down somewhere that layouts mustn't change, tags shouldn't be reused after they get deprecated, etc., the fact that nobody would notice if you do will encourage people to silently do it anyway.
I agree that we don't want to have all of them clutter a single .rst file once there are hundreds or thousands, but there are ways to organize that better which we can decide on once we get to that point.
participants (9)
-
Dan Handley -
Ilias Apalodimas -
Jose Marinho -
Julius Werner -
Rob Herring -
Simon Glass -
Stuart Yoder -
Tom Rini -
Varun Wadekar