Skip to main content

Python CXL Host

CXL Host is capable of acquiring more memory capacity by utilizing CXL-attached memory devices. These devices can be single logical device (SLD) or multi-logical device (MLD). This mechanism allows the CXL host to map CXL-attached device memory (HDM) into its own memory address space.

Alt text

Starting Components​

In the root directory of opencxl-core:

./cxl-util start <options>
options:
  -c, --comp TEXT     Components. e.g. "-c fm -c switch ..."   [required]
  --config-file TEXT  <Config File> input path.
  --log-file TEXT     <Log File> output path.
  --pcap-file TEXT    <Packet Capture File> output path.
  --log-level TEXT    Specify log level.
  --show-timestamp    Show timestamp.
  --show-loglevel     Show log level.
  --show-linenumber   Show line number.
  --help              Show this message and exit.
./cxl-util start \
  -c fm \
  -c switch \
  -c host-group \
  -c sld-group \
  --config-file configs/1vcs_4sld.yaml

Note: When running CXL Switch (switch), CXL Host-Group (host-group), or CXL SLD-Group (sld-group), you must specify --config-file.

Example 2: Run Multiple Components In Multiple Terminals​

Terminal 1

./cxl-util start \
  -c fm \
  -c switch \
  --config-file configs/1vcs_4sld.yaml

Terminal 2

./cxl-util start \
  -c host-group \
  -c sld-group \
  --config-file configs/1vcs_4sld.yaml

The specified configuration file must be the same between terminals.

note

While there are use cases for running components in separate terminals, it is highly recommended that you run all components in a single terminal for better flow visualization.

MLD and Fabric Manager​

The above examples run Single Logical Device (SLD) which can serve only one host at a time.

Working with the CXL Fabric Manager, Multi Logical Devices (MLD) can be used to serve multiple hosts at the same time. This is accomplished by CXL Fabric Manager sending commands to the CXL switch to bind/unbind an individual LD within the MLD device to/from a particular host.

Example​

./cxl-util start \
  -c fm \
  -c switch \
  -c host-group \
  -c mld-group \
  --config-file configs/2vcs_1mld.yaml

The above command will start with MLD's LD 0 to LD 3 bound to the first host and LD 4 to LD 7 bound to the second host.

Unbind​

./cxl-util fm unbind <VCS ID> <vPPB ID>
./cxl-util fm unbind 0 0
[FabricManagerSocketIoServer] Received SocketIO Request: vcs:unbind, payload: {'virtualCxlSwitchId': 0, 'vppbId': 0}
Host unbind notification root_port 0, device_vppb 0, val 0
[CxlVirtualSwitch 0] Started unbinding physical port from vPPB 0
[ConfigSpaceManager:DSP1] Stopping ConfigSpaceManager
[DownstreamPortDevice:DSP1] Unbinding LD-ID 0 to vPPB3 (noop)
VPPB unbinded from physical port, type: CXL_COMPONENT_TYPE.DSP
[CxlVirtualSwitch 0] Succcessfully unbound physical port from vPPB 0
[SYS-SW] Received Unbinding Root Port: 0, vPPB: 0 from FM
[SYS-SW] FM unbind device @ port: 0
[CxlHostPort0:CacheController] Removing MemoryRange addr: 0x10000000 addr_type: CFG
[CxlHostPort0:CacheController] Removing MemoryRange addr: 0xfe200000 addr_type: MMIO
[CxlHostPort0:CacheController] Removing MemoryRange addr: 0x100000000000 addr_type: CXL_UNCACHED
[FabricManagerSocketIoServer] Response: {'error': '', 'result': 'SUCCESS'}
Received Host confirmation for Unbinding Root Port: 0, vPPB: 0 from host (root port=0)

Bind​

./cxl-util fm bind <VCS ID> <vPPB ID> <Physical Port ID> [optional:<LD-ID>]
./cxl-util fm bind 0 0 1 0
[FabricManagerSocketIoServer] Received SocketIO Request: vcs:bind, payload: {'virtualCxlSwitchId': 0, 'vppbId': 0, 'physicalPortId': 1, 'ldId': 0}
[CxlVirtualSwitch 0] Started Binding physical port 1 to vPPB 0
[DownstreamPortDevice:DSP1] Binding LD-ID 0 to vPPB0
[CxlVirtualSwitch 0] Succcessfully bound physical port 1 to vPPB 0 with LD-ID 0
Host bind notification root_port 0, device_vppb 0, val 1
[FabricManagerSocketIoServer] Response: {'error': '', 'result': 'SUCCESS'}
[SYS-SW] Received Binding Root Port: 0, vPPB: 0, val 1 from FM, vPPB: 0
[CxlHostPort0:CacheController] Adding MemoryRange addr: 0x50000000 addr_type: CFG
[CxlHostPort0:CacheController] Adding MemoryRange addr: 0xfea00000 addr_type: MMIO
[CxlHostPort0:CacheController] Adding MemoryRange addr: 0x100040000000 addr_type: CXL_UNCACHED

note

While the above example demos MLD use case, the same demo could be run for SLD as well.