KMAC DV document
Goals
- DV
- Verify all KMAC IP features by running dynamic simulations with a SV/UVM based testbench
- Develop and run all tests based on the testplan below towards closing code and functional coverage on the IP and all of its sub-modules
- FPV
- Verify TileLink device protocol compliance with an SVA based testbench
Current status
Design features
For detailed information on KMAC design features, please see the KMAC HWIP technical specification.
Testbench architecture
KMAC testbench has been constructed based on the CIP testbench architecture.
Block diagram
Top level testbench
Top level testbench is located at hw/ip/kmac/dv/tb/tb.sv
. It instantiates the KMAC DUT module hw/ip/kmac/rtl/kmac.sv
.
In addition, it instantiates the following interfaces, connects them to the DUT and sets their handle into uvm_config_db
:
- Clock and reset interface
- TileLink host interface
- KMAC IOs
- Interrupts (
pins_if
) - Devmode (
pins_if
)
Common DV utility components
The following utilities provide generic helper tasks and functions to perform activities that are common across the project:
Compile-time configurations
Two compile-time configurations are tested:
kmac_masked
- this configuration enables internal secure data masking featureskmac_unmasked
- this configuration disables all internal security features
A macro define EN_MASKING
is defined as a build option in hw/ip/kmac/dv/kmac_base_sim_cfg.hjson
, which is used to set the correct compile-time settings.
Global types & methods
All common types and methods defined at the package level can be found in
kmac_env_pkg
. Some of them in use are:
parameter int KMAC_NUM_SHARES = 2;
parameter int KMAC_NUM_KEYS_PER_SHARE = 16;
parameter bit [TL_AW-1:0] KMAC_FIFO_BASE = 32'h800;
parameter bit [TL_AW-1:0] KMAC_FIFO_END = 32'FFC;
// interrupt types
typedef enum int {
KmacDone = 0,
KmacFifoEmpty = 1,
KmacErr = 2;
KmacNumIntrs = 3
} kmac_intr_e;
// types of application interfaces
typedef enum int {
AppKeymgr,
AppLc,
AppRom
} kmac_app_e;
typedef virtual pins_if#(1) idle_vif;
typedef virtual kmac_sideload_if sideload_vif
// Helper function that returns the KMAC key size in bytes
function automatic int get_key_size_bytes(kmac_pkg::key_len_e len);
case (len)
Key128: return 16;
Key192: return 24;
Key256: return 32;
Key384: return 48;
Key512: return 64;
default: `uvm_fatal("kmac_env_pkg", $sformatf("%0s is an invalid key length", len.name()))
endcase
endfunction
TL_agent
KMAC testbench instantiates (already handled in CIP base env) tl_agent which provides the ability to drive and independently monitor random traffic via TL host interface into KMAC device.
EDN Agent
The KMAC testbench instantiates a push_pull_agent
in Pull
mode as the agent modelling the EDN interface (this is already handled in the CIP base classes).
This agent will return random data as entropy after a random delay any time the KMAC sends a request.
KMAC_APP Agent
The KMAC testbench instantiates an array of kmac_app_agent
to model the application interfaces used by other IP blocks to request a KMAC hash operation on some data.
These interfaces are used to send in message data to the KMAC, and to receive an output digest.
UVM RAL Model
The KMAC RAL model is created with the ralgen
FuseSoC generator script automatically when the simulation is at the build stage.
It can be created manually by invoking regtool
:
Reference models
The KMAC testbench utilizes a C++ reference model for various hashing operations (SHA3, SHAKE, CSHAKE, KMAC) to check the DUT’s digest output for correctness.
Stimulus strategy
Test sequences
All test sequences reside in hw/ip/kmac/dv/env/seq_lib
.
The kmac_base_vseq
virtual sequence is extended from cip_base_vseq
and serves as a starting point.
All test sequences are extended from kmac_base_vseq
.
It provides commonly used handles, variables, functions and tasks that the test sequences can simple use / call.
Some of the most commonly used tasks / functions are as follows:
set_prefix()
- This task encodes 2 input strings (function name and customization string) into a bytestream as per NIST standard specifications and writes the data to thePREFIX
CSRswrite_msg()
- This task breaks down the input messages into chunks less than or equal to the TLUL bus size, and writes each chunk to the message FIFO window of the KMACread_digest_shares()
- This task reads the output digest data from theSTATE_SHARE
windows, manually squeezing (polling for more output data) as necessary
Functional coverage
To ensure high quality constrained random stimulus, it is necessary to develop a functional coverage model. Please refer to the covergroups section under testplan for coverpoints that are implemented.
Self-checking strategy
Scoreboard
The kmac_scoreboard
is primarily used for end to end checking.
It creates the following analysis ports to retrieve the data monitored by corresponding interface agents:
- tl_a_chan_fifo: TL address channel
- tl_d_chan_fifo: TL data channel
- kmac_app_req_fifo[kmac_pkg::NumAppIntf]: An array of analysis FIFOs to hold request transactions coming from the various application interfaces
- kmac_app_rsp_fifo[kmac_pkg::NumAppIntf]: An array of analysis FIFOs to hold response transactions coming from the various application interfaces
- edn_fifo: FIFO used to hold transactions coming from the EDN interface
The KMAC scoreboard implements a cycle-accurate model of the DUT that is used to thoroughly check the operation of the KMAC IP. Though complex to implement, this is extremely useful for CSR prediction as many of the status fields rely on exact timing of the design (e.g. CSR fields that reflect the internal message FIFO pointers need to be predicted in the scoreboard on the exact cycle that they are updated in the design otherwise it will result in prediction mismatches).
The cycle-accurate model is also designed to check the operation of the various application interfaces as well as the EDN interface to provide full confidence in the design’s correctness.
In addition to the cycle-accurate model, the scoreboard tracks the input message as it is written to the message FIFO in chunks, assembles it into a full bytestream, and uses this complete message as input for the DPI-C++ reference model to produce the expected digest value. As the test sequence reads the output STATE_SHARE windows after a hash operation, the scoreboard will assemble the read digest data into the actual digest value, which can easily be compared against the output of the C++ reference model.
Assertions
- TLUL assertions: The
tb/kmac_bind.sv
binds thetlul_assert
assertions to the IP to ensure TileLink interface protocol compliance. - Unknown checks on DUT outputs: The RTL has assertions to ensure all outputs are initialized to known values after coming out of reset.
Building and running tests
We are using our in-house developed regression tool for building and running our tests and regressions. Please take a look at the link for detailed information on the usage, capabilities, features and known issues. Here’s how to run a smoke test:
$ $REPO_TOP/util/dvsim/dvsim.py $REPO_TOP/hw/ip/kmac/dv/kmac_sim_cfg.hjson -i kmac_smoke
Testplan
Testpoints
Milestone | Name | Tests | Description |
---|---|---|---|
V1 | smoke | _smoke | KMAC smoke test will contain a number of rounds, and acts as a base for many other tests. In each round, we run a full KMAC hashing operation:
This test, and all other tests in the testplan, will be checked for correctness in two different ways:
|
V1 | csr_hw_reset | kmac_csr_hw_reset | Verify the reset values as indicated in the RAL specification.
|
V1 | csr_rw | kmac_csr_rw | Verify accessibility of CSRs as indicated in the RAL specification.
|
V1 | csr_bit_bash | kmac_csr_bit_bash | Verify no aliasing within individual bits of a CSR.
|
V1 | csr_aliasing | kmac_csr_aliasing | Verify no aliasing within the CSR address space.
|
V1 | csr_mem_rw_with_rand_reset | kmac_csr_mem_rw_with_rand_reset | Verify random reset during CSR/memory access.
|
V1 | regwen_csr_and_corresponding_lockable_csr | kmac_csr_rw kmac_csr_aliasing | Verify regwen CSR and its corresponding lockable CSRs.
Note:
This is only applicable if the block contains regwen and locakable CSRs. |
V1 | mem_walk | kmac_mem_walk | Verify accessibility of all memories in the design.
|
V1 | mem_partial_access | kmac_mem_partial_access | Verify partial-accessibility of all memories in the design.
|
V2 | long_msg_and_output | _long_msg_and_output | Same as the smoke test, except with a message of up to 100KB.
Max firmware input size for KMAX is around 392KB, but this is too large for a DV
simulation.
Average input size from firmware would be around 60-100KB, so we use 100KB as a max
input size for DV, but will enable easy extensibility for emulation testing where we can
enable much larger messages.
Allow output length to vary up to 1KB (for XOF functions).
If output length is greater than |
V2 | burst_write | _burst_write | This is the same as the long_message test, except we burst-write chunks of the message into the msg_fifo, and disable intermediate status/CSR checks. |
V2 | test_vectors | _test_vectors_sha3_224 _test_vectors_sha3_256 _test_vectors_sha3_384 _test_vectors_sha3_512 _test_vectors_shake_128 _test_vectors_shake_256 _test_vectors_kmac _test_vectors_kmac_xof | These tests drive NIST test vectors for SHA3/SHAKE/KMAC into the design and check the output against the expected digest values. |
V2 | sideload | _sideload | Same as the smoke test, except we set cfg.sideload and provide a valid sideloaded key as well as a valid SW-provided key. KMAC should operate on the sideloaded key. |
V2 | app | _app | Test that the Keymgr/ROM/LC can all initiate a KMAC operation through the application interface - Keymgr uses KMAC hash, while ROM/LC use CShake. Use an array of kmac_app_agents to send message data to the KMAC and to control the hashing logic, and set cfg.sideload if the Keymgr is enabled. The result digest sent to the kmac_app_agent will be compared against the result from the DPI reference model. In addition, read from the STATE window afterwards and confirm that this access is blocked and will return 0. |
V2 | app_with_partial_data | _app_with_partial_data | Basd on the kmac_app test, this test will send partial data from application interface
by sending |
V2 | error | _error | Try several error sequences:
// TODO: Not all of these errors have been supported yet, and more errors might be // added later. // So this might be split into several error tests later on. |
V2 | key_error | _key_error | Test kmac responses correctly when keymgr app sends request but key is not valid. Stimulus:
Check:
|
V2 | edn_timeout_error | _edn_timeout_error | Test kmac responses correctly when EDN response timeout.
Based on kmac app sequence, this sequence write
|
V2 | entropy_mode_error | _entropy_mode_error | Test kmac responses correctly when entropy mode is configured incorrectly.
Based on kmac edn_timeout sequence, this sequence write incorrect
|
V2 | lc_escalation | _lc_escalation | Randomly set Checks:
|
V2 | intr_test | kmac_intr_test | Verify common intr_test CSRs that allows SW to mock-inject interrupts.
|
V2 | alert_test | kmac_alert_test | Verify common
|
V2 | tl_d_oob_addr_access | kmac_tl_errors | Access out of bounds address and verify correctness of response / behavior |
V2 | tl_d_illegal_access | kmac_tl_errors | Drive unsupported requests via TL interface and verify correctness of response / behavior. Below error cases are tested bases on the [TLUL spec]({{< relref "hw/ip/tlul/doc/_index.md#explicit-error-cases" >}})
|
V2 | tl_d_outstanding_access | kmac_csr_hw_reset kmac_csr_rw kmac_csr_aliasing kmac_same_csr_outstanding | Drive back-to-back requests without waiting for response to ensure there is one transaction outstanding within the TL device. Also, verify one outstanding when back- to-back accesses are made to the same address. |
V2 | tl_d_partial_access | kmac_csr_hw_reset kmac_csr_rw kmac_csr_aliasing kmac_same_csr_outstanding | Access CSR with one or more bytes of data. For read, expect to return all word value of the CSR. For write, enabling bytes should cover all CSR valid fields. |
V2S | shadow_reg_update_error | kmac_shadow_reg_errors | Verify shadowed registers' update error.
|
V2S | shadow_reg_read_clear_staged_value | kmac_shadow_reg_errors | Verify reading a shadowed register will clear its staged value.
|
V2S | shadow_reg_storage_error | kmac_shadow_reg_errors | Verify shadowed registers' storage error.
|
V2S | shadowed_reset_glitch | kmac_shadow_reg_errors | Verify toggle shadowed_rst_n pin can trigger storage error.
|
V2S | shadow_reg_update_error_with_csr_rw | kmac_shadow_reg_errors_with_csr_rw | Run shadow_reg_update_error sequence in parallel with csr_rw sequence.
|
V2S | tl_intg_err | kmac_tl_intg_err kmac_sec_cm | Verify that the data integrity check violation generates an alert.
|
V2S | sec_cm_bus_integrity | Verify the countermeasure(s) BUS.INTEGRITY. | |
V2S | sec_cm_lc_escalate_en_intersig_mubi | Verify global LC_ESCALATE_EN mubi | |
V2S | sec_cm_sw_key_key_masking | Verify the countermeasure(s) SW_KEY.KEY.MASKING. | |
V2S | sec_cm_key_sideload | Verify the key from KeyMgr is sideloaded. | |
V2S | sec_cm_cfg_shadowed_config_shadow | Verify the countermeasure(s) CFG_SHADOWED.CONFIG.SHADOW. | |
V2S | sec_cm_fsm_sparse | Verify the countermeasure(s) FSM.SPARSE. | |
V2S | sec_cm_ctr_redun | Verify the countermeasure(s) CTR.REDUN. | |
V2S | sec_cm_cfg_shadowed_config_regwen | Verify the countermeasure(s) CFG_SHADOWED.CONFIG.REGWEN. | |
V2S | sec_cm_fsm_global_esc | Verify the countermeasure(s) FSM.GLOBAL_ESC. | |
V2S | sec_cm_fsm_local_esc | Verify the countermeasure(s) FSM.LOCAL_ESC. | |
V2S | sec_cm_logic_integrity | Verify the countermeasure(s) LOGIC.INTEGRITY. | |
V3 | entropy_timers | _entropy | Test entropy interface for KMAC. This test randomly chooses to execute either a SW-controlled hash or a hash from the App interface. All configuration fields are left as is, but now we set both of the internal timer CSRs:
This will be checked in the scoreboard using the cycle acurate model. |
V3 | throughput | _throughput | Measure the throughput of the various hashing calculations and make sure they correspond to the expected throughput range for the design. |
V3 | stress_all_with_rand_reset | kmac_stress_all_with_rand_reset | This test runs 3 parallel threads - stress_all, tl_errors and random reset. After reset is asserted, the test will read and check all valid CSR registers. |
Covergroups
Name | Description |
---|---|
app_cg | Covers several scenarios related to the app interface:
Note that this covergroup will be duplicated once per app interface. |
cmd_process_cg | Covers that the KMAC can handle seeing a
|
config_cg | Covers that all valid configuration settings for the KMAC have been tested. Individual config settings that will be covered include:
|
edn_cg | Covers that EDN entropy can be received by KMAC while keccak rounds are active/inactive, crossed with a few entropy configuration values (fast entropy, etc...). |
error_cg | Covers all error scenarios:
|
msg_len_cg | Covers various input message length ranges, to ensure that KMAC can operate successfully on different sized messages. The minimum tested msg length is 0 bytes, and the maximum length is 10_000 bytes, we will cover that an acceptable distribution of lengths has been seen, and specifically cover some corner cases (like length 0). |
msgfifo_level_cg | Covers that all possible fifo statuses have been seen when running different hash operations (like sha3/shake/cshake/kmac), such as various fifo depths, fifo full, and fifo empty. |
msgfifo_write_mask_cg | Covers that the msgfifo has been written using all possible TLUL masks. |
output_digest_len_cg | Similar to the |
prefix_range_cg | The prefix used for CSHAKE and KMAC (function_name + customization_string) are only allowed to be valid alphabet letters, or a space character. This covergroup covers that all of these characters have appeared in a prefix value. |
regwen_val_when_new_value_written_cg | Cover each lockable reg field with these 2 cases:
This is only applicable if the block contains regwen and locakable CSRs. |
sha3_status_cg | Covers that all sha3-related status fields have eventually been seen. |
sideload_cg | Covers that the KMAC sees scenarios where the sideloaded key is provided and should be
used ( |
state_read_mask_cg | Covers that the state windows have been read using all possible TLUL masks. |
tl_errors_cg | Cover the following error cases on TL-UL bus:
|
tl_intg_err_cg | Cover all kinds of integrity errors (command, data or both) and cover number of error bits on each integrity check. Cover the kinds of integrity errors with byte enabled write on memory if applicable: Some memories store the integrity values. When there is a subword write, design re-calculate the integrity with full word data and update integrity in the memory. This coverage ensures that memory byte write has been issued and the related design logic has been verfied. |