How Transparent Bridge(TB) works?
TB provides electrical isolation between PCI buses. For TB, the Configuration Status Register(CSR) with “type 1” header inform CPU to keep enumerating beyond this bridge since there are additional devices lie downstream. EP devices with “type 0” header in their CSR’s inform the enumerator such as BIOS or CPU that no more devices lie downstream. These CSR’s include BARs used to request memory or IO apertures from the host.
What is NTB?
In addition to electrical isolation, the NTB adds logical isolation by providing processor domain partitioning and address translation between memory-mapped spaces of these domains. With NTB, the devices on either side of the bridge are not visible from the other side, but a path is provided for data transfer and status exchange between the processor domains.
Address translation:
In the NTB environment, PCIe devices need to translate the addresses that cross from one memory space to the other. Each NTB port has two sets of BARs, one for the primary side and other for the secondary side. BARs are used to define address translating windows into the memory space on the other side of the NTB and allow the transactions to be mapped to to local memory or I/Os. Each BAR has a setup register which defines the size and type of the window and an address translation register. While TB’s forward all CSRs based on bus number, NTB’s only accept CSR transactions addressed to the device itself.
There are two translation techniques.
- direct-address: add an offset to the BAR in which the transaction terminates.
- lookup-table-based: mapping local addresses to host bus addresses as the location of the index field within the address is programmable to adjust window size. The index is used to provide the upper bits for the new memory location.
Inter-processor communication
The NTB also allow hosts on each side of the bridge to exchange indormation about the status through scratchpad registers, doorbell registers, and heartbeat messages.
Scratchpad Registers: These r/w from both sides of the NTB.
Doorbell Registers: used to send interrupt. These are software controlled interrupt request registers with associated masking registers for each interface on the NTB. These registers can be accessed from both sides.
Heartbeat Messages: are sent from primary to the secondary host to indicate it is still alive. The secondary host monitors the state of the primary host and takes appropriate action upon detection of the failure.
In summary, the NTB provides a powerful features for the people wants to implement dual host, dual fabric, fail-over and load sharing capability to their systems. Meaning, high availability systems.
my tech journals 