fork() Causes DMA Buffer in Physical Memory to Retain Stale Data on Subsequent Writes describes an issue with a forked process causing issues with memory for DMA. It wasn't clear what mechanism was being used to allocate the memory for DMA.
The test_dma_descriptor_credits program was modified with the option of spawning a child process to see if could replicate the issue when VFIO is used for DMA mappings.
Used the following FPGA designs, which exercise both the memory mapped and stream interface test functions:
- dma_blkram (TEF1001)
- XCKU5P_DUAL_QSFP_dma_stream_loopback
- Nitefury Project-0 version 0x2
Using a 4.18.0-553.51.1.el8_10.x86_64 Kernel.
Spawned the child process, using the heap for mappings, no DMA mapping access and the test passes:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f no_dma_mapping_access
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 35492
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 35493
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 35494
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 35495
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 35496
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 35497
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Test: PASS
Spawned the child process, using the heap for mappings, reading DMA mappings. The test passed, and the child reads all the DMA mappings as zeros:
mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f read_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 37648
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 37649
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37650
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37651
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37652
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 37653
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Test: PASS
[mr_halfword@skylake-alma coverage]$
Spawned the child process, using the heap for mappings, toggling DMA mappings. The test passed, and the child read all the DMA mappings as zeros prior to the toggling:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 37738
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 37739
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37740
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37742
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 37743
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 37744
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Test: PASS
Spawned the child process, using the heap for mappings, writing DMA mappings. The test passed, and the child read all the DMA mappings as zeros prior to the writing:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f write_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 38350
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 38351
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38352
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38353
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38356
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 38357
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Test: PASS
Spawned the child process, using shared memory for mappings, reading DMA mappings. The test passed, and the child reads all the DMA mappings as having non-zero content:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f read_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 38434
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0xffffffff
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 38435
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38436
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38439
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38441
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 38443
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0xffffffff
Test: PASS
Spawned the child process, using shared memory for mappings, toggling DMA mappings. The test fails, and the child reads all the DMA mappings as having non-zero content prior to the toggling:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 38688
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Test timeout waiting for H2C descriptors to complete (started 5 completed 0 channel_status 0x10)
card_words[1] actual=0xc3910ca0 expected=0x3c6ef35f
Test: FAIL
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 38699
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 38706
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 38712
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 38714
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Test: FAIL
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 38731
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Test timeout waiting for H2C descriptors to complete (started 5 completed 0 channel_status 0x10)
card_words[1] actual=0xc3910ca0 expected=0x3c6ef35f
Test: FAIL
Spawned the child process, using shared memory for mappings, writing DMA mappings. The test passes, and the child reads all the DMA mappings as having non-zero content prior to the writing:
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f write_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 38901
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Child has 2362152 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 38902
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38904
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38906
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38907
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 38908
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Child has 25055055 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0x00000000
Test: PASS
Spawned the child process, using shared memory for mappings, toggling DMA mappings and using MADV_DONTFORK on the DMA mapping prior to the fork. The test passes, while the child gets a SIGSEGV attempting to access the DMA mappings (expected due to use of MADV_DONTFORK):
[mr_halfword@skylake-alma coverage]$ xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping -b shared_memory -m
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 25
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 85
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 38959
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
SIGSEGV in child si_code=1 si_addr=0x7f51b041f000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 38960
SIGSEGV in child si_code=1 si_addr=0x7f51b040d000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38961
SIGSEGV in child si_code=1 si_addr=0x7f51b040d000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38963
SIGSEGV in child si_code=1 si_addr=0x7f51b040d000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 38964
SIGSEGV in child si_code=1 si_addr=0x7f51b040d000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 38965
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
SIGSEGV in child si_code=1 si_addr=0x7f51b03ff000
Test: PASS
Using a 5.14.21-150500.55.62-default Kernel, this behaves in the same way as AlmaLinux 8.10:
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f no_dma_mapping_access
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3897
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3898
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3899
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3900
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3901
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3902
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f read_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3911
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3912
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3913
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3914
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3915
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3916
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child reading words 0x00000000 0x00000000 0x00000000
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3925
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3926
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3927
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3928
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3929
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3930
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f write_dma_mapping
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3939
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has all 2428928 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3940
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3941
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3942
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3943
Child has all 25219072 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3944
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has all 25210880 bytes of 3 DMA mappings as zeros
Child toggling words 0x00000000 0x00000000 0x00000000
Child has 12 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xffffffff 0xffffffff 0xffffffff
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f read_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3954
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0xffffffff
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3955
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3956
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3957
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3958
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0x00000000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3959
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child reading words 0xad4b0002 0x00000000 0xffffffff
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3968
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Test timeout waiting for H2C descriptors to complete (started 5 completed 0 channel_status 0x10)
card_words[1] actual=0xc3910ca0 expected=0x3c6ef35f
Test: FAIL
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3969
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 3970
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 3971
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Hello from child pid 3972
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Test timeout waiting for C2H descriptor to complete (processed 0 completed 0 channel_status 0x1)
Test timeout waiting for H2C descriptors to complete (started 1 completed 0 channel_status 0x10)
Failed after 0 out of 3069 messages transferred by DMA
Test: FAIL
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3973
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Test timeout waiting for H2C descriptors to complete (started 5 completed 0 channel_status 0x10)
card_words[1] actual=0xc3910ca0 expected=0x3c6ef35f
Test: FAIL
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f write_dma_mapping -b shared_memory
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3983
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
Child has 2362151 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Child has 2362152 out of 2428928 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0x00000000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3984
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3985
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3986
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3987
Child has 15298 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0x00000000
Child has 15307 out of 25219072 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0xffffffff
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 3988
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
Child has 25055054 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0xad4b0002 0x00000000 0xffffffff
Child has 25055055 out of 25210880 bytes in 3 DMA mappings with non-zero values
Child toggling words 0x52b4fffd 0xffffffff 0x00000000
Test: PASS
linux@skylake-alma:~/fpga_sio/software_tests/eclipse_project/bin/coverage> xilinx_dma_bridge_for_pcie/test_dma_descriptor_credits -f toggle_dma_mapping -b shared_memory -m
Opening device 0000:15:00.0 (10ee:7024) with IOMMU group 41
Enabled bus master for 0000:15:00.0
Opening device 0000:2d:00.0 (10ee:9038) with IOMMU group 85
Enabled bus master for 0000:2d:00.0
Opening device 0000:36:00.0 (10ee:7011) with IOMMU group 86
Enabled bus master for 0000:36:00.0
Testing DMA bridge bar 0 memory size 0x120000
Hello from child pid 3997
Testing 1179648 bytes of card memory, using rings with 1023 descriptors, and a total of 288 descriptors
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e26000
Test: PASS
Testing DMA bridge bar 2 AXI Stream
Device 0000:2d:00.0 design XCKU5P_DUAL_QSFP_dma_stream_loopback routes updated
Hello from child pid 3998
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e14000
Successfully sent 3069 messages from Ch1->0 with a total of 7847433 words in 9198 descriptors
Hello from child pid 3999
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e14000
Successfully sent 3069 messages from Ch0->1 with a total of 7847433 words in 9198 descriptors
Hello from child pid 4000
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e14000
Successfully sent 3069 messages from Ch3->2 with a total of 7847433 words in 9198 descriptors
Hello from child pid 4001
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e14000
Successfully sent 3069 messages from Ch2->3 with a total of 7847433 words in 9198 descriptors
Test: PASS
Testing DMA bridge bar 2 memory size 0x40000000
Hello from child pid 4002
Testing 12570624 bytes of card memory, using rings with 1023 descriptors, and a total of 3069 descriptors
SIGSEGV in child si_code=1 si_addr=0x7ff0f3e06000
Test: PASS