Openness-Score/docs/methodology.md

Dasharo Openness Score methodology

Dasharo Openness Score calculation methodology is different for UEFI images and coreboot-based images mainly due to their different structure, but also for calculation simplicity.

Calculating Dasharo Openness Score for UEFI images

Axiom: UEFI images are always assumed to be vendor firmware images, thus the methodology assumes no open-source code.

The utility makes decisions based on the report produced by UEFIExtract. The calculation flow looks as follows:

1. Look for the entries of type Region and save their properties for later. Also separate the BIOS Region which will be used later.

2. Start classifying entries:

   • Take all non-BIOS regions and classify them as either closed-source group or data regions, based on known region types. For unknown region types, classify it as closed-source region.
   • Take all entries in the BIOS region and extract UEFI Firmware Volumes from them. Create new uefi.UEFIVolume instances, which trigger the UEFI Volume calculations described in section Calculating metrics for UEFI Firmware Volumes. Save all instances in the array for future calculations.
   • Take all empty paddings between UEFI Firmware Volumes in BIOS region and between regions and add them to the empty region group.
   • Take all non-empty paddings between UEFI Firmware Volumes in BIOS region and between regions and add them to the data region group.

3. Sum up the size of all region groups: empty, data and closed-source.

4. Sum up the size of all groups from all UEFIVolume instances saved in the array previously: empty, data and closed-source.

5. Check if the sums from point 3 and 4 give the total size of the image. If yes, do nothing. If not add the difference to the data group and print an error.

6. Export the data to markdown and pie charts using UEFIVolume.export_markdown.

Calculating metrics for UEFI Firmware Volumes

Axiom: Some UEFI Firmware Volumes may cotnain nested volumes that are compressed. Compressed entries i nthe UEFIExtract report do not have the base address of the components, just uncompressed size. To simplify the calculations and reliably present the data on charts, compressed UEFI Firmware Volumes are classified as closed-source as a whole.

Whenever a new instance of UEFIVolume is created, the utility performs the following steps:

1. Parse the report entries and extract only those belonging to given volume.

2. Create a new instance of UEFIVolume for each nested uncompressed UEFI Firmware Volume (FFSv2). This triggers the whole UEFI Firmware Volume calculation procedure recursively.

3. Start classifying entries:

   • Take all empty paddings and volume free space inside the current UEFI Firmware Volume and add them to the empty group.
   • Take all non-empty paddings inside the current UEFI Firmware Volume and add them to the data group.
   • As the UEFI Firmware Volume is comprised of executable code mainly or blobs, we do not classify anything else.

4. Sum up the size of all groups classified up to now: empty, data and closed-source (there is currently no closed-source classified yet, so it will be 0).

5. Sum up the size of all groups from all nested UEFIVolume instances: empty, data and closed-source.

6. Check if given UEFI Firmware Volume is an NVAR storage using UEFIVolume._is_nvar_store_volume. The volume is considered an NVAR storage if at least 90% (currently defined as 90% by UEFIVolume.NVAR_VOLUME_THRESHOLD) of the volume entries are NVAR entry.

7. If the volume is an NVAR storage, the difference between the volume size and classified entries size (so the unclassified bytes) are added up to the data group. Otherwise, if the volume is not NVAR storage, it must contain executabel code, thus the difference is counted as closed-source.

At this point the UEFI Firmware Volume metrics are ready to be included in total calculations in step 4 of Calculating Dasharo Openness Score for UEFI images.

Calculating Dasharo Openness Score for coreboot images

coreboot images are processed with cbfstool and the output of cbfstool is parsed by the utility to make decisions about the image components. The calculation flow looks as follows:

1. Parse the flashmap layout printed by cbfstool and extract regions' attributes.

2. For each flashmap region containing CBFS, create a new instance of coreboot.CBFSImage and save it to an array. Creating new instance of CBFSImage triggers the CBFS image calculations described in section Calculating metrics for CBFS images.

3. Start classifying regions:

   • Skip regions containing CBFSes, they are calcululated inside CBFSImage instances
   • Flashmap regions, which names are found in coreboot.DasharoCorebootImage.CODE_REGIONS, classify as open-source. Currently the only region applicable is BOOTBLOCK. Some architectures do not store bootblock as a CBFS file but as flashmap region.
   • Flashmap regions, which names are found in coreboot.DasharoCorebootImage.BLOB_REGIONS (where regions known to contain blobs are defined), classify as closed-source.
   • Flashmap regions, which names are found in coreboot.DasharoCorebootImage.EMPTY_REGIONS, classify as empty. Currently only the UNUSED flashmap region name applies.
   • Flashmap regions, which name is found in coreboot.DasharoCorebootImage.DATA_REGIONS (where regions known to contain data only are defined), classify as data.
   • Flashmap regions, which names are found in coreboot.DasharoCorebootImage.SKIP_REGIONS are not classified due to being cotnainers or aliases to other regions. Counting them would result in duplication of the sizes when calculating metrics. These are standard flashmap region names aggregating smaller, nested regions.
   • If any region does not apply to above rules, save it in a separate array of uncategorized regions. It will be counted as closed-source in next step.

4. Sum up the size of all region groups: empty, data, open-source and closed-source. Add the sum of uncategorized regions to closed-source.

5. Sum up the size of all groups from all CBFSImage instances saved in the array previously: empty, data, open-source and closed-source.

6. Check if the first flashmap region offset in flash starts at 0. If yes, then do nothing. Otherwise count all bytes from 0 to the given offset as closed-source (we don't know what lies before the first flashmap region, it can be ME or something else).

7. Check if all classified components sizes (empty, data, open-source and closed-source) calculated in point 4-6 sum up to the full image size. If not, print a warning.

8. Export the data to markdown and pie charts using coreboot.DasharoCorebootImage.export_markdown.

Calculating metrics for CBFS images

Axiom: coreboot's payload is always assumed to be open-source (with a small exception known to Dasharo images which sometimes include EFI drivers for Ethernet NICs).

CBFS regions are processed with cbfstool and the output of cbfstool is parsed by the utility to make decisions about the CBFS components. The calculation flow looks as follows:

1. Parse the CBFS content printed by cbfstool and extract CBFS files' attributes.

2. Extract the config file from CBFS and parse the build options used to produce given CBFS.

3. Check if iPXE has been included:

   • Check for EDK2_ENABLE_IPXE Kconfig value. If set, save the information that iPXE was built for EFI and included in EUFI payload.
   • If EDK2_ENABLE_IPXE was not set, check for PXE Kconfig vlaue. If set iPXE was added as a legacy OptionROM.
     • Check for PXE_ROM Kconfig value. If set, it means the iPXE was included as an external binary. It will be later counted as closed-source as we don't know its origin. If PXE_ROM was not set, it means it was built from source.
     • If iPXE was built from source, extract the PXE_ROM_ID Kconfig value and save the PCI ID used for the iPXE CBFS file name. If PXE_ROM_ID not present, assume default 10ec,8168.

4. Check if external Ethernet NIC EFI driver was included:

   • Check if EDK2_LAN_ROM_DRIVER Kconfig value is set. If not, go to step 5.
   • If yes, extract the fallback/payload from the CBFS and use UEFIExtract on it to extract the file with coreboot.CBFSImage.DASHARO_LAN_ROM_GUID.
   • Compress the extracted EFI driver with LZMA to estimate the compressed size.
   • Save the size of compressed EFI driver for later calculations.

5. Start classifying CBFS files:

   • CBFS files which type is found in coreboot.CBFSImage.OPEN_SOURCE_FILETYPES and names are not found in coreboot.CBFSImage.CLOSED_SOURCE_EXCEPTIONS classify as open-source.
   • CBFS files of type raw, which names are found in coreboot.CBFSImage.RAW_OPEN_SOURCE_FILES classify as open-source.
   • If the iPXE was detected to be built from source and included as legacy OptionROM CBFS file, classify pci<PXE_ROM_ID>.rom as open-source.
   • CBFS files which names are found in coreboot.CBFSImage.CLOSED_SOURCE_FILETYPES, classify as closed-source. or with CBFS file's type found in
   • CBFS files which type is found in CBFSImage.OPEN_SOURCE_FILETYPES and name is found in CBFSImage.CLOSED_SOURCE_EXCEPTIONS, classify as closed-source.
   • CBFS files of type raw which names are found in coreboot.CBFSImage.RAW_CLOSED_SOURCE_FILES, classify as close-source.
   • CBFS files with type null classify as empty.
   • CBFS files which type is found in coreboot.CBFSImage.DATA_FILETYPES classify as data.
   • CBFS files of type raw and names found in coreboot.CBFSImage.RAW_DATA_FILES classify as data.
   • CBFS files not applying to above rules should be save to an array of uncategorized files. They will be counted as closed-source code in next steps because we were unable to identify what can be inside.

6. Sum up the size of all file groups: empty, data, open-source and closed-source. Add the sum of uncategorized files to closed-source.

7. If an external Ethernet NIC EFI driver was detected, subtract the previously saved compressed EFI driver size from open-source and add it to closed-source group.

8. Check for truncated CBFS:

   • vboot RW CBFSes are often truncated from empty space and the cbfstool does not print empty space file at the end of CBFS. Check if the difference between last file offset + its size and the CBFS image size is bigger than 64 bytes. If yes, count the difference as empty. The 64 bytes is the size of metadata following the end of CBFSfile.
   • cbfstool prints the size of the files in CBFS, but does not account for file's metadata size. Sum up the size of all file groups: empty, data, open-source and closed-source. Then subtract it from the CBFS region size to obtain metadata size. Add the metadata size to data group. It will also ensure that the CBFS file groups sum up to whole CBFS size.

At this point the CBFS image metrics are ready to be included in total calculations in step 5 of Calculating Dasharo Openness Score for coreboot images.