A PCIe Non-Transparent Bridge (NTB) is a hardware feature that allows two independent systems - typically two CPUs or SoCs- to communicate over a PCIe link without merging their memory spaces. Unlike a standard PCIe hierarchy, where one root complex controls multiple endpoints, NTB enables peer-to-peer communication, fault isolation, and high-bandwidth data exchange while keeping each system’s memory, OS, and PCIe domain independent.

For embedded systems and product developers, PCIe NTB matters because it combines the speed of PCIe with the safety and modularity required in high-reliability designs. Whether you build industrial controllers, network appliances, or automotive platforms, NTB offers a robust way to scale compute, implement redundancy, or connect multiple boards in a single product.

How PCIe NTB Works

1. Independent Root Complexes

In a typical PCI Express system, only one root complex controls the PCIe fabric.

In contrast, an NTB allows both connected devices to act as independent root complexes. Each side sees the other as a PCIe “endpoint,” even though both are hosts.

This avoids:

  • shared memory conflicts,
  • OS-level resource contention,
  • dependency on a single root complex.

2. Memory Windows

NTBs expose memory windows that map regions from one host’s memory into the address space of the other. This memory is not unified - it's explicitly mapped and controlled.

Typical memory window features:

  • configurable aperture sizes,
  • doorbells (interrupt-like signals),
  • scratchpads (small shared registers),
  • translation and access control.

These mechanisms allow fast, deterministic data transfer without requiring unified addressing.

3. Doorbells and Scratchpads

NTBs implement lightweight signaling:

  • Doorbells: hardware-triggered interrupts used for event notifications, queue updates, or synchronization.
  • Scratchpads: small registers for status exchange, heartbeats, or configuration data.

Together, they create an IPC (inter-process communication) mechanism with microsecond-level latency.

4. DMA Engines

NTB implementations often support DMA transfers, enabling zero-copy movement of large data blocks between hosts.

Engineers use this for:

  • packet buffers,
  • sensor data streams,
  • shared log buffers,
  • replication or failover data.

5. Software Stacks

Linux includes a mature NTB subsystem supporting:

  • ntb_hw drivers for Intel, AMD, Microchip, NXP platforms,
  • ntb_transport for queue-based data exchange,
  • peer-to-peer messaging APIs.

Many vendors also provide proprietary stacks optimized for storage or networking.

When to Use PCIe NTB

PCIe NTB shines whenever you need fast, modular, and isolated communication between two compute units.

1. High Availability (HA) and Failover Systems

In industrial automation, medical devices, telecom, and automotive ECUs, NTB is used to implement dual-host redundancy.

Advantages:

  • each CPU runs its own OS instance,
  • state is replicated via NTB DMA,
  • if one unit fails, the other continues operation,
  • no single point of failure in the PCIe tree.


This makes NTB a strong choice for IEC 61508, ISO 26262, or medical safety-class designs.

2. Multi-Board Compute Expansion

Embedded products often hit CPU limitations. NTB allows designers to link two boards (or two SoMs) and:

  • scale CPU cores without redesigning a monolithic system,
  • offload compute-heavy tasks to a second processor,
  • create specialized processing modules (AI inference, networking, encryption).

Unlike Ethernet-based communication, NTB offers PCIe bandwidth and latency, ideal for data-heavy workloads.

3. Storage Clustering

Datacenter storage appliances use NTB for:

  • dual-controller NVMe storage,
  • multipath redundancy,
  • metadata synchronization,
  • low-latency failover.

For embedded edge storage appliances or NAS devices, NTB enables enterprise-grade high availability.

4. Network Appliances and Firewalls

Networking vendors use NTB for:

  • dual-CPU routing
  • TCP offload
  • packet mirroring
  • shared flow tables

When porting these concepts into embedded industrial routers or IoT gateways, NTB allows high-speed coordination between main and auxiliary processors.

5. FPGA–CPU Co-Processing

Many FPGAs support NTB-like modes, enabling:

  • CPU <-> FPGA high-bandwidth channels,
  • shared ring buffers,
  • real-time accelerator offloading.

This is valuable in:

  • machine vision,
  • robotics,
  • industrial motion control,
  • automotive sensor fusion.

Industry Applications

Automotive

ECUs that require redundancy, such as ADAS compute platforms, use PCIe NTB to maintain system integrity. NTB provides:

  • high-speed replication between compute nodes,
  • fault-tolerant communication,
  • deterministic behavior independent of OS load.

This is particularly relevant as vehicles move toward domain controllers and integrated architectures.

Industrial Automation & Robotics

In PLCs, safety controllers, or multi-axis robots:

  • NTB synchronizes critical state data
  • enables redundant CPUs
  • supports low-latency command sharing

Applications requiring SIL or fail-operational behavior benefit directly.

IoT Gateways & Edge Computing

Edge devices often combine:

  • a general-purpose SoC
  • an accelerator SoC or FPGA


NTB allows them to coordinate without merging PCIe domains, keeping each module secure and isolated.

Medical Devices

In medical imaging or patient-monitoring systems:

  • NTB supports failover
  • allows modular compute upgrades
  • maintains strict segregation between subsystems

Compliance requires predictable behavior and isolation - properties NTBs deliver inherently.

PCIe NTB vs Alternatives

NTB vs Ethernet/RDMA

Feature

PCIe NTB

Ethernet/RDMA
Latency

Extremely low (microseconds)

Higher (10-100 µs)
Bandwidth

PCIe line rate

Depends on NIC

Root complex independence

 Yes

N/A

Isolation

 Strong

Network-dependent
Scalability

Limited (# of hosts)

 Very high

Use NTB when you need two tightly coupled systems.

Use Ethernet when you need scalability or distributed systems.

NTB vs PCIe Peer-to-Peer

Peer-to-peer still requires a single root complex.

NTB removes this requirement and provides true dual-host independence.

NTB vs Shared Memory Buses

Shared memory requires:

  • unified address spaces,
  • tightly synchronized firmware,
  • complex arbitration logic.

NTB avoids this at the cost of a more complex PCIe topology.

Best Practices for Using PCIe NTB

Architectural Tips

  • Define clear ownership of data regions.
  • Use doorbells for event-driven synchronization.
  • Validate DMA boundaries to avoid corruption.
  • Segment memory windows per function to reduce contention.
  • Implement heartbeats using scratchpads.

Performance Tips

  • Keep DMA buffers aligned to cache-line boundaries.
  • Use larger memory windows for streaming workloads.
  • Enable PCIe ASPM carefully—low-power states may increase latency.

Reliability Tips

  • Implement failover logic in firmware, not only in Linux.
  • Use CRCs or consistency checks for replicated state.
  • Monitor NTB link health continuously.

Common Mistakes

Assuming NTB works like shared memory

It doesn’t. Memory mapping is explicit and limited.

Underestimating synchronization complexity

Doorbells and scratchpads require careful design.

Ignoring PCIe topology constraints

NTB-capable hardware is specific; you can't enable NTB arbitrarily.

Using tiny memory windows for large transfers

Leads to bottlenecks. Use DMA for bulk data.

FAQ

Is PCIe NTB supported on Linux?

Yes. Linux includes drivers, NTB transport, and both Intel/AMD/Microchip implementations.

Can NTB connect more than two hosts?

Generally no. NTB is designed for two-host architectures.

Does NTB work over PCIe switches?

Yes, if the switch supports NTB partitioning or routing.

Is NTB suitable for safety-certified systems?

Yes. Its isolation properties make it attractive for SIL/ASIL designs.

Conclusion

PCIe NTB provides one of the most effective ways to connect two independent compute systems with high bandwidth, low latency, and strict isolation. It is ideal for applications where reliability, modularity, and deterministic communication are critical - from industrial automation to automotive safety controllers and medical devices.

For engineering teams designing fault-tolerant embedded platforms or scaling compute across multiple boards, NTB offers a proven and robust architectural building block. Conclusive Engineering’s expertise in firmware development, board design, and high-reliability embedded systems makes us well-positioned to help you integrate PCIe NTB into your next-generation product.