Xilinx 7 Series SymbiFlow Partial Reconfiguration Flow¶
Note: SymbiFlow currently does not support partial bitstream generation. This is a goal in the future, but at the moment partial FASM must be concatenated with an overlay to generate a full bitstream.
Background¶
Partition Regions¶
In this documentation the terms partition region and region of interest (ROI) are used interchangeably to refer to some smaller portion of a larger FPGA architecture. This region may or may not align with frame boundaries, but the most tested use-case is for partition regions that are one clock region tall.
Overlay Architecture¶
The overlay architecture is essentially the “inverse” of all the partition regions in a design; it includes everything in the full device that is not in a partition region. Typically this includes chip IOs and the PS region if the chip has one.
Synthetic IO Tiles (Synth IOs)¶
Synthetic IO tiles are “fake” IOs inserted into the partition region architecture so VPR will route top level IOs to a specific graph node. This method allows partition region architectures to interface with each other and the overlay.
Vivado Node vs Wire¶
A wire is a small electrically connected part of the FPGA contained within a single tile. A Vivado node is an electrically connected collection of wires that can span multiple tiles.
Flow Overview¶
A simplified view of the partition region flow is as follows:
- Define each partition region architecture
- Define the overlay architecture based on the partition regions chosen
- Build each architecture separately
- Map a top level verilog file to each architecture
- Generate FASM for each partition region and the overlay
- Concatenate FASM for each architecture together and generate final bitstream
Partition Region Example (switch_processing)¶
This example contains two partition regions that are each about the size of one clock region.
The goal of this test is to have two partition regions with identical interfaces so switch “data” can be passed through each region before being displayed on LEDs. Each partition region can then have an arbitrary module mapped to it that processes the data in some way before the output. The example modules used currently are an add_1 module, a blink module, and an identity module.
Define the first partition region:
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr1-roi-virt/design.json
{
"info":
{
"name": "pr1",
"GRID_X_MAX": 55,
"GRID_X_MIN": 10,
"GRID_Y_MAX": 51,
"GRID_Y_MIN": 0
},
"ports": [
{
"name": "clk",
"type": "clk",
"node": "CLK_HROW_TOP_R_X60Y130/CLK_HROW_CK_BUFHCLK_L0",
"wire": "HCLK_L_X57Y130/HCLK_CK_BUFHCLK0",
"pin": "SYN0"
},
{
"name": "in[0]",
"type": "in",
"node": "INT_L_X0Y124/EE2BEG0",
"pin": "SYN1"
},
{
"name": "in[1]",
"type": "in",
"node": "INT_L_X0Y125/SE6BEG0",
"pin": "SYN2"
},
{
"name": "in[2]",
"type": "in",
"node": "INT_R_X1Y117/SE2BEG1",
"pin": "SYN3"
},
{
"name": "in[3]",
"type": "in",
"node": "INT_L_X0Y116/EE2BEG0",
"pin": "SYN4"
},
{
"name": "out[0]",
"type": "out",
"node": "INT_L_X2Y103/SE6BEG0",
"pin": "SYN5"
},
{
"name": "out[1]",
"type": "out",
"node": "INT_L_X4Y100/SE6BEG0",
"pin": "SYN6"
},
{
"name": "out[2]",
"type": "out",
"node": "INT_L_X2Y104/SS6BEG2",
"pin": "SYN7"
},
{
"name": "out[3]",
"type": "out",
"node": "INT_L_X2Y104/SS6BEG0",
"pin": "SYN8"
},
{
"name": "rst",
"type": "in",
"node": "INT_R_X21Y119/EE4BEG2",
"pin": "SYN9"
}
]
}
Here we see the info section defines the boundaries of the partition region. It is important to use the prjxray grid, not the VPR grid or the Vivado grid, to define these boundaries. The ports section is then used to define the interface pins for the region. A synth IO will be placed to correspond to each of these interface pins. Each pin must contain a name, pin name, type, and node name. The name and pin name must be unique identifiers. The type can be in, out or clk. The node is the vivado node that a synth IO should be connected to.
Optionally, a wire name can be provided to give an exact location for the synth IO. If a wire is not provided it will be inferred as the first wire outside of the partition region on the given node. Providing an explicit wire name is especially important when using nodes that cross all the way through the partition region, such as clock nodes.
Now the CMake files must be defined properly for the first partition region architecture:
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr1-roi-virt/CMakeLists.txt
add_xc_device_define_type(
ARCH artix7
DEVICE xc7a50t-arty-switch-processing-pr1
ROI_DIR ${symbiflow-arch-defs_SOURCE_DIR}/xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr1-roi-virt
TILE_TYPES
CLBLL_L
CLBLL_R
CLBLM_L
CLBLM_R
BRAM_L
PB_TYPES
SLICEL
SLICEM
BRAM_L
)
The important argument here is ROI_DIR
which points to the directory containing the design.json
defined earlier.
Next, define the second partition region in a similar way as the first:
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr2-roi-virt/design.json
{
"info":
{
"name": "pr2",
"GRID_X_MAX": 57,
"GRID_X_MIN": 10,
"GRID_Y_MAX": 156,
"GRID_Y_MIN": 105
},
"ports": [
{
"name": "clk",
"type": "clk",
"node": "CLK_HROW_BOT_R_X60Y26/CLK_HROW_CK_BUFHCLK_L8",
"wire": "HCLK_CLB_X56Y26/HCLK_CLB_CK_BUFHCLK8",
"pin": "SYN0"
},
{
"name": "in[0]",
"type": "in",
"node": "INT_L_X20Y51/SS2BEG0",
"pin": "SYN1"
},
{
"name": "in[1]",
"type": "in",
"node": "INT_R_X1Y34/EE4BEG3",
"pin": "SYN2"
},
{
"name": "in[2]",
"type": "in",
"node": "INT_L_X0Y47/EE4BEG3",
"pin": "SYN3"
},
{
"name": "in[3]",
"type": "in",
"node": "INT_L_X0Y39/EE4BEG1",
"pin": "SYN4"
},
{
"name": "out[0]",
"type": "out",
"node": "INT_L_X20Y49/ER1BEG_S0",
"pin": "SYN5"
},
{
"name": "out[1]",
"type": "out",
"node": "INT_R_X3Y34/WW4BEG2",
"pin": "SYN6"
},
{
"name": "out[2]",
"type": "out",
"node": "INT_L_X2Y33/WW2BEG2",
"pin": "SYN7"
},
{
"name": "out[3]",
"type": "out",
"node": "INT_L_X4Y30/WW4BEG2",
"pin": "SYN8"
},
{
"name": "rst",
"type": "in",
"node": "INT_R_X23Y46/WW4BEG3",
"pin": "SYN9"
}
]
}
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr2-roi-virt/CMakeLists.txt
add_xc_device_define_type(
ARCH artix7
DEVICE xc7a50t-arty-switch-processing-pr1
ROI_DIR ${symbiflow-arch-defs_SOURCE_DIR}/xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-pr1-roi-virt
TILE_TYPES
CLBLL_L
CLBLL_R
CLBLM_L
CLBLM_R
BRAM_L
PB_TYPES
SLICEL
SLICEM
BRAM_L
)
The last design.json
that must be defined is for the overlay. It is mostly a list of the json for the partition regions contained in the design. One important change is the pin names must still be unique across all ports in the overlay. Any explicit wires must also be changed to be on the other side of the partition region boundary.
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-overlay-virt/design.json
[
{
"info":
{
"name": "pr1",
"GRID_X_MAX": 57,
"GRID_X_MIN": 10,
"GRID_Y_MAX": 51,
"GRID_Y_MIN": 0
},
"ports": [
{
"name": "clk",
"type": "clk",
"node": "CLK_HROW_TOP_R_X60Y130/CLK_HROW_CK_BUFHCLK_L0",
"wire": "HCLK_L_X57Y130/HCLK_CK_BUFHCLK0",
"pin": "SYN0"
},
{
"name": "in[0]",
"type": "in",
"node": "INT_L_X0Y124/EE2BEG0",
"pin": "SYN1"
},
{
"name": "in[1]",
"type": "in",
"node": "INT_L_X0Y125/SE6BEG0",
"pin": "SYN2"
},
{
"name": "in[2]",
"type": "in",
"node": "INT_R_X1Y117/SE2BEG1",
"pin": "SYN3"
},
{
"name": "in[3]",
"type": "in",
"node": "INT_L_X0Y116/EE2BEG0",
"pin": "SYN4"
},
{
"name": "out[0]",
"type": "out",
"node": "INT_L_X2Y103/SE6BEG0",
"pin": "SYN5"
},
{
"name": "out[1]",
"type": "out",
"node": "INT_L_X4Y100/SE6BEG0",
"pin": "SYN6"
},
{
"name": "out[2]",
"type": "out",
"node": "INT_L_X2Y104/SS6BEG2",
"pin": "SYN7"
},
{
"name": "out[3]",
"type": "out",
"node": "INT_L_X2Y104/SS6BEG0",
"pin": "SYN8"
},
{
"name": "rst",
"type": "in",
"node": "INT_L_X0Y119/EE4BEG1",
"pin": "SYN9"
}
]
},
{
"info":
{
"name": "pr2",
"GRID_X_MAX": 57,
"GRID_X_MIN": 10,
"GRID_Y_MAX": 156,
"GRID_Y_MIN": 105
},
"ports": [
{
"name": "clk",
"type": "clk",
"node": "CLK_HROW_BOT_R_X60Y26/CLK_HROW_CK_BUFHCLK_L8",
"wire": "HCLK_CLB_X56Y26/HCLK_CLB_CK_BUFHCLK8",
"pin": "SYN10"
},
{
"name": "in[0]",
"type": "in",
"node": "INT_L_X20Y51/SS2BEG0",
"pin": "SYN11"
},
{
"name": "in[1]",
"type": "in",
"node": "INT_R_X1Y34/EE4BEG3",
"pin": "SYN12"
},
{
"name": "in[2]",
"type": "in",
"node": "INT_L_X0Y47/EE4BEG3",
"pin": "SYN13"
},
{
"name": "in[3]",
"type": "in",
"node": "INT_L_X0Y39/EE4BEG1",
"pin": "SYN14"
},
{
"name": "out[0]",
"type": "out",
"node": "INT_L_X20Y49/ER1BEG_S0",
"pin": "SYN15"
},
{
"name": "out[1]",
"type": "out",
"node": "INT_R_X3Y34/WW4BEG2",
"pin": "SYN16"
},
{
"name": "out[2]",
"type": "out",
"node": "INT_L_X2Y33/WW2BEG2",
"pin": "SYN17"
},
{
"name": "out[3]",
"type": "out",
"node": "INT_L_X4Y30/WW4BEG2",
"pin": "SYN18"
},
{
"name": "rst",
"type": "in",
"node": "INT_R_X23Y46/WW4BEG3",
"pin": "SYN19"
}
]
}
]
xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-overlay-virt/CMakeLists.txt
add_xc_device_define_type(
ARCH artix7
DEVICE xc7a50t-arty-switch-processing-overlay
OVERLAY_DIR ${symbiflow-arch-defs_SOURCE_DIR}/xc/xc7/archs/artix7/devices/xc7a50t-arty-switch-processing-overlay-virt
TILE_TYPES
CLBLL_L
CLBLL_R
CLBLM_L
CLBLM_R
BRAM_L
LIOPAD_M
LIOPAD_S
LIOPAD_SING
RIOPAD_M
RIOPAD_S
RIOPAD_SING
CLK_BUFG_BOT_R
CLK_BUFG_TOP_R
CMT_TOP_L_UPPER_T
CMT_TOP_R_UPPER_T
HCLK_IOI3
PB_TYPES
SLICEL
SLICEM
BRAM_L
IOPAD
IOPAD_M
IOPAD_S
BUFGCTRL
PLLE2_ADV
HCLK_IOI3
)
The important argument here is OVERLAY_DIR
which points to the directory containing the design.json
for this overlay. Notice this CMakeLists.txt
also contains more tile/pb types because it contains the real IOs.
Continuing on past design.json
definitions, CMake needs to be informed these new architectures should be built. This is done in another CMakeLists.txt
by adding the following:
xc/xc7/archs/artix7/devices/CMakeLists.txt
add_xc_device_define(
ARCH artix7
PART xc7a50tfgg484-1
USE_ROI
DEVICES xc7a50t-arty-switch-processing-pr1 xc7a50t-arty-switch-processing-pr2
)
add_xc_device_define(
ARCH artix7
PART xc7a50tfgg484-1
USE_OVERLAY
DEVICES xc7a50t-arty-switch-processing-overlay
)
The last step before switching over to adding a test is adding to boards.cmake
:
add_xc_board(
BOARD arty-switch-processing-pr1
DEVICE xc7a50t-arty-switch-processing-pr1
PACKAGE test
PROG_CMD "${OPENOCD} -f ${PRJXRAY_DIR}/utils/openocd/board-digilent-basys3.cfg -c \\\"init $<SEMICOLON> pld load 0 \${OUT_BIN} $<SEMICOLON> exit\\\""
PART xc7a35tcsg324-1
)
add_xc_board(
BOARD arty-switch-processing-pr2
DEVICE xc7a50t-arty-switch-processing-pr2
PACKAGE test
PROG_CMD "${OPENOCD} -f ${PRJXRAY_DIR}/utils/openocd/board-digilent-basys3.cfg -c \\\"init $<SEMICOLON> pld load 0 \${OUT_BIN} $<SEMICOLON> exit\\\""
PART xc7a35tcsg324-1
)
add_xc_board(
BOARD arty-switch-processing-overlay
DEVICE xc7a50t-arty-switch-processing-overlay
PACKAGE test
PROG_CMD "${OPENOCD} -f ${PRJXRAY_DIR}/utils/openocd/board-digilent-basys3.cfg -c \\\"init $<SEMICOLON> pld load 0 \${OUT_BIN} $<SEMICOLON> exit\\\""
PART xc7a35tcsg324-1
)
This defines a separate board for each of the partition regions and overlay so they can be mapped to separately.
Now to define a test. This part of the documentation will not go in detail on how to define a new test case in symbiflow-arch-defs, but will point out items of importance for using the partial reconfiguration flow.
All of the following snippets are from xc/xc7/tests/switch_processing/CMakeLists.txt
Here the add_1 and blink modules are mapped to pr1 and pr2 respectively. The identity function is then also mapped to each partition region.
Here the overlay verilog is mapped to the overlay architecture. This overlay verilog connects switches to the input of the first partition region, connects the output of the first partition region to the input of the second partition region, and then connects the output of the second partition region to LEDs.
Lastly, multiple merged bitstream targets are defined. These targets will concatenate the FASM generated by each included target and produce the final bitstream. By varying which targets are included different functionality is created without having to remap any new regions after it has been done once. Just concatenate the resulting FASM and get different functionality.
The last thing to cover related to the SymbiFlow partial reconfiguration flow is synthetic ibufs and obufs required in the overlay verilog:
switch_processing_arty_overlay.v
Currently the SYN_IBUF
and SYN_OBUF
must be explicitly defined for each top level IO that will be constrained to a synth IO. In the future this should be able to be resolved using a yosys io map pass, but currently if explicit synthetic buffers are not defined the top level IOs will be packed into a real IO. This will prevent constraining the top level IOs to the intended synthetic IO location.
The overlay pcf file can then be written to constrain real IOs to chip IOs and synthetic IOs to synthetic IOs.
Frequently Encountered Errors¶
Error | Solution |
SYN-IOPAD unroutable | Make sure the chosen node is driven in the correct direction for the I/O type it is being used as. Inputs to a partition region must be driven from outside the partition region and outputs must be driven from inside the partition region. |