Hi Matt,
I added the –mem-size 8GB and there are some other messages and errors showing
up now.
MIOpen(HIP): Warning [ParseAndLoadDb] File is unreadable:
/opt/rocm-4.0.1/miopen/share/miopen/db/gfx801100.HIP.fdb.txt
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
MIOpen Error: /root/driver/MLOpen/src/ocl/convolutionocl.cpp:150: Invalid
filter channel number
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
MIOpen Error: /root/driver/MLOpen/src/ocl/convolutionocl.cpp:150: Invalid
filter channel number
MIOpen Error: 3 at
/home/dfong/work/ext_ips/gem5-apu-cu256-dnn/gem5/gem5-resources/src/gpu/DNNMark/core/include/dnn_utility.h1057Ticks:
264369621500
David
docker run --rm -v ${PWD}:${PWD} -v
${PWD}/gem5/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0
-w ${PWD} gcr.io/gem5-test/gcn-gpu:v21-2 gem5/build/GCN3_X86/gem5.opt
gem5/configs/example/apu_se.py --mem-size 8GB --num-compute-units 256 -n3
--benchmark-root=gem5/gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_conv
-cdnnmark_test_fwd_conv --options="-config
gem5/gem5-resources/src/gpu/DNNMark/config_example/conv_config.dnnmark -mmap
gem5/gem5-resources/src/gpu/DNNMark/mmap.bin" |& tee
gem5_apu_cu256_8GB_run_dnnmark_test_fwd_conv_40latency.log
Global frequency set at 1000000000000 ticks per second
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (5) does not divide
range [1:75] into equal-sized buckets. Rounding up.
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (2) does not divide
range [1:10] into equal-sized buckets. Rounding up.
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (2) does not divide
range [1:64] into equal-sized buckets. Rounding up.
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (10000) does not
divide range [1:1e+06] into equal-sized buckets. Rounding up.
. . .
Forcing maxCoalescedReqs to 32 (TLB assoc.)
Forcing maxCoalescedReqs to 32 (TLB assoc.)
build/GCN3_X86/base/remote_gdb.cc:381: warn: Sockets disabled, not accepting
gdb connections
tcmalloc: large alloc 1073741824 bytes == 0x55f2039c2000 @ 0x7f2bdb141680
0x7f2bdb161ff4 0x55f1bb897441 0x55f1bbf47e53 0x55f1bb316617 0x7f2bdb609718
0x7f2bdb609afb 0x7f2bdb609dc0 0x7f2bdb3d5d6d 0x7f2bdb3ddef6 0x7f2bdb52becb
0x7f2bdb6090f4 0x7f2bdb3d5d6d 0x7f2bdb3ddef6 0x7f2bdb52becb 0x7f2bdb52c252
0x7f2bdb52c63f 0x7f2bdb530c81 0x7f2bdb5c0527 0x7f2bdb3d5d6d 0x7f2bdb3d746d
0x7f2bdb3e106b 0x7f2bdb609810 0x55f1bb92ed14 0x55f1ba9956f6 0x7f2bda4db0b3
0x55f1ba9b604e
warn: dir_cntrl0.memory is deprecated. The request port for Ruby memory output
to the main memory is now called `memory_out_port`
warn: system.ruby.network adopting orphan SimObject param 'ext_links'
warn: system.ruby.network adopting orphan SimObject param 'int_links'
warn: failed to generate dot output from m5out/config.dot
build/GCN3_X86/sim/simulate.cc:194: info: Entering event queue @ 0. Starting
simulation...
build/GCN3_X86/mem/ruby/system/Sequencer.cc:573: warn: Replacement policy
updates recently became the responsibility of SLICC state machines. Make sure
to setMRU() near callbacks in .sm files!
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
gem5 version 21.2.1.0
gem5 compiled Mar 10 2022 22:54:39
gem5 started Mar 11 2022 16:50:21
gem5 executing on 09498c9eca4f, pid 1
command line: gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py
--mem-size 8GB --num-compute-units 256 -n3
--benchmark-root=gem5/gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_conv
-cdnnmark_test_fwd_conv '--options=-config
gem5/gem5-resources/src/gpu/DNNMark/config_example/conv_config.dnnmark -mmap
gem5/gem5-resources/src/gpu/DNNMark/mmap.bin'
info: Standard input is not a terminal, disabling listeners.
Num SQC = 64 Num scalar caches = 64 Num CU = 256
incrementing idx on 4
incrementing idx on 8
incrementing idx on 12
. . .
incrementing idx on 248
incrementing idx on 252
"dot" with args ['-Tsvg', '/tmp/tmp1io1e4wn'] returned code: 1
stdout, stderr:
b''
b'Error: /tmp/tmp1io1e4wn: syntax error in line 236909 scanning a quoted string
(missing endquote? longer than 16384?)\nString
starting:"clk_domain=system.ruby.clk_domain \\eventq_index=0 \\latency=1\n'
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
. . .
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall
set_robust_list(...)
build/GCN3_X86/sim/syscall_emul.cc:85: warn: ignoring syscall rt_sigaction(...)
(further warnings will be suppressed)
build/GCN3_X86/sim/syscall_emul.cc:85: warn: ignoring syscall
rt_sigprocmask(...)
(further warnings will be suppressed)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall get_mempolicy(...)
build/GCN3_X86/arch/generic/debugfaults.hh:145: warn: MOVNTDQ: Ignoring
non-temporal hint, modeling as cacheable!
build/GCN3_X86/arch/x86/generated/exec-ns.cc.inc:27: warn: instruction
'frndint' unimplemented
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:704: warn: unimplemented
ioctl: AMDKFD_IOC_ACQUIRE_VM
build/GCN3_X86/sim/syscall_emul.hh:1862: warn: mmap: writing to shared mmap
region is currently unsupported. The write succeeds on the target, but it will
not be propagated to the host or shared mappings
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:455: warn: Signal events are
only supported currently
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/power_state.cc:105: warn: PowerState: Already in the
requested power state, request ignored
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall
set_robust_list(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:599: warn: unimplemented
ioctl: AMDKFD_IOC_SET_SCRATCH_BACKING_VA
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:609: warn: unimplemented
ioctl: AMDKFD_IOC_SET_TRAP_HANDLER
build/GCN3_X86/sim/syscall_emul.hh:2081: warn: prlimit: unimplemented resource 7
build/GCN3_X86/sim/syscall_emul.hh:2081: warn: prlimit: unimplemented resource 7
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
sh: 1: Cannot fork
MIOpen(HIP): Error [ValidateGcnAssemblerImpl] Specified assembler does not
support AMDGPU. Expect performance degradation.
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
. . .
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall fdatasync(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall fdatasync(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall fdatasync(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall fdatasync(...)
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
MIOpen(HIP): Warning [ParseAndLoadDb] File is unreadable:
/opt/rocm-4.0.1/miopen/share/miopen/db/gfx801100.HIP.fdb.txt
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
MIOpen Error: /root/driver/MLOpen/src/ocl/convolutionocl.cpp:150: Invalid
filter channel number
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
build/GCN3_X86/sim/syscall_emul.cc:683: warn: fcntl: unsupported command 6
MIOpen Error: /root/driver/MLOpen/src/ocl/convolutionocl.cpp:150: Invalid
filter channel number
MIOpen Error: 3 at
/home/dfong/work/ext_ips/gem5-apu-cu256-dnn/gem5/gem5-resources/src/gpu/DNNMark/core/include/dnn_utility.h1057Ticks:
264369621500
Exiting because exiting with last active thread context
From: Matt Sinclair <[email protected]>
Sent: Thursday, March 10, 2022 6:02 PM
To: gem5 users mailing list <[email protected]>
Cc: David Fong <[email protected]>; Kyle Roarty <[email protected]>; Matthew
Poremba <[email protected]>
Subject: Re: [gem5-users] Re: gem5 : X86 + GCN3 (gfx8001) + test_fwd_conv
Just to be clear: —mem-size is an input arg for the apu_se.py script.
Matt
Sent from my iPhone
On Mar 10, 2022, at 7:44 PM, Matt Sinclair via gem5-users
<[email protected]<mailto:[email protected]>> wrote:
I am on my phone and thus cannot easily look at the line that failed at the
moment, but my first step would be to increase the size of the memory gem5 is
assuming — try —mem-size=8GB or 16GB and let us know if that solves the problem.
Matt
Sent from my iPhone
On Mar 10, 2022, at 5:12 PM, David Fong via gem5-users
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I’m trying to run test_fwd_conv for gem5 with X86 CPU and GCN3 (gfx801) APU
with 256 CU using git with gem5 v21.2.1.0
Linux> cd gem5/gem5-resources/src/gpu/DNNMark
Linux> docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID
gcr.io/gem5-test/gcn-gpu:v21-2 ./setup.sh HIP
Linux> docker run --rm -v ${PWD}:${PWD} -w ${PWD}/build -u $UID:$GID
gcr.io/gem5-test/gcn-gpu:v21-2 make
Linux> docker run --rm -v ${PWD}:${PWD}
-v${PWD}/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD}
gcr.io/gem5-test/gcn-gpu:v21-2 python3 generate_cachefiles.py cachefiles.csv
--gfx-version=gfx801 --num-cus=256
Linux> mv gem5/gem5-resources/src/gpu/DNNMark/cachefiles/gfx801_256.ukdb
gem5/gem5-resources/src/gpu/DNNMark/cachefiles/gfx801100.ukdb
Linux> vim gem5/build_opts/GCN3_X86
NUMBER_BITS_PER_SET = '256'
Linux> cd gem5
Linxu> docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID
gcr.io/gem5-test/gcn-gpu:v21-2 scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
Linux> cd ../../../../
linux> docker run --rm -v ${PWD}:${PWD} -v
${PWD}/gem5/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0
-w ${PWD} gcr.io/gem5-test/gcn-gpu:v21-2 gem5/build/GCN3_X86/gem5.opt
gem5/configs/example/apu_se.py --num-compute-units 256 -n3
--benchmark-root=gem5/gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_conv
-cdnnmark_test_fwd_conv --options="-config
gem5/gem5-resources/src/gpu/DNNMark/config_example/conv_config.dnnmark -mmap
gem5/gem5-resources/src/gpu/DNNMark/mmap.bin"
An error message occurred for the test:
HIP Error at
/home/dfong/work/ext_ips/gem5-apu-cu256-dnn/gem5/gem5-resources/src/gpu/DNNMark/core/include/data_manager.h49
hipErrorOutOfMemory
How to fix this error ?
David
MESSAGES SHORTENED
Global frequency set at 1000000000000 ticks per second
build/GCN3_X86/mem/mem_interface.cc:791: warn: DRAM device capacity (8192
Mbytes) does not match the address range assigned (512 Mbytes)
. . .
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (10000) does not
divide range [1:1.6e+06] into equal-sized buckets. Rounding up.
build/GCN3_X86/base/stats/storage.hh:279: warn: Bucket size (10000) does not
divide range [1:1.6e+06] into equal-sized buckets. Rounding up.
Forcing maxCoalescedReqs to 32 (TLB assoc.)
Forcing maxCoalescedReqs to 32 (TLB assoc.)
Forcing maxCoalescedReqs to 32 (TLB assoc.)
. . .
Forcing maxCoalescedReqs to 32 (TLB assoc.)
build/GCN3_X86/base/remote_gdb.cc:381: warn: Sockets disabled, not accepting
gdb connections
warn: dir_cntrl0.memory is deprecated. The request port for Ruby memory output
to the main memory is now called `memory_out_port`
warn: system.ruby.network adopting orphan SimObject param 'ext_links'
warn: system.ruby.network adopting orphan SimObject param 'int_links'
warn: failed to generate dot output from m5out/config.dot
build/GCN3_X86/sim/simulate.cc:194: info: Entering event queue @ 0. Starting
simulation...
build/GCN3_X86/mem/ruby/system/Sequencer.cc:573: warn: Replacement policy
updates recently became the responsibility of SLICC state machines. Make sure
to setMRU() near callbacks in .sm files!
gem5 Simulator System.
http://gem5.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__gem5.org&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OkH-8nM02VdNPRt_miVO36vI9580zW1SgNQ4MzWRfqc&m=7r3w2XxzFbRgIeC6-XIHvihzeamlJkbmqVL5aEB9c_w&s=QTsN9sgXhwR4_EmFelH8kQ-b_SiIyy5a0wBfKhyf00g&e=>
gem5 is copyrighted software; use the --copyright option for details.
gem5 version 21.2.1.0
gem5 compiled Mar 10 2022 21:44:19
gem5 started Mar 10 2022 22:25:08
gem5 executing on 84084e0cba7d, pid 1
command line: gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py
--num-compute-units 256 -n3
--benchmark-root=gem5/gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_conv
-cdnnmark_test_fwd_conv '--options=-config
gem5/gem5-resources/src/gpu/DNNMark/config_example/conv_config.dnnmark -mmap
gem5/gem5-resources/src/gpu/DNNMark/mmap.bin'
info: Standard input is not a terminal, disabling listeners.
Num SQC = 64 Num scalar caches = 64 Num CU = 256
incrementing idx on 4
incrementing idx on 8
incrementing idx on 12
. . .
incrementing idx on 248
incrementing idx on 252
"dot" with args ['-Tsvg', '/tmp/tmp7b3e5gva'] returned code: 1
stdout, stderr:
b''
b'Error: /tmp/tmp7b3e5gva: syntax error in line 236909 scanning a quoted string
(missing endquote? longer than 16384?)\nString
starting:"clk_domain=system.ruby.clk_domain \\eventq_index=0 \\latency=1\n'
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
. . .
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall
set_robust_list(...)
build/GCN3_X86/sim/syscall_emul.cc:85: warn: ignoring syscall rt_sigaction(...)
(further warnings will be suppressed)
build/GCN3_X86/sim/syscall_emul.cc:85: warn: ignoring syscall
rt_sigprocmask(...)
(further warnings will be suppressed)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall get_mempolicy(...)
build/GCN3_X86/arch/generic/debugfaults.hh:145: warn: MOVNTDQ: Ignoring
non-temporal hint, modeling as cacheable!
build/GCN3_X86/arch/x86/generated/exec-ns.cc.inc:27: warn: instruction
'frndint' unimplemented
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/mem_state.cc:443: info: Increasing stack size by one page.
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:704: warn: unimplemented
ioctl: AMDKFD_IOC_ACQUIRE_VM
build/GCN3_X86/sim/syscall_emul.hh:1862: warn: mmap: writing to shared mmap
region is currently unsupported. The write succeeds on the target, but it will
not be propagated to the host or shared mappings
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:455: warn: Signal events are
only supported currently
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/sim/power_state.cc:105: warn: PowerState: Already in the
requested power state, request ignored
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall
set_robust_list(...)
build/GCN3_X86/sim/syscall_emul.cc:74: warn: ignoring syscall mprotect(...)
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:599: warn: unimplemented
ioctl: AMDKFD_IOC_SET_SCRATCH_BACKING_VA
build/GCN3_X86/gpu-compute/gpu_compute_driver.cc:609: warn: unimplemented
ioctl: AMDKFD_IOC_SET_TRAP_HANDLER
HIP Error at
/home/dfong/work/ext_ips/gem5-apu-cu256-dnn/gem5/gem5-resources/src/gpu/DNNMark/core/include/data_manager.h49
hipErrorOutOfMemory
Ticks: 116736880500
Exiting because exiting with last active thread context
_______________________________________________
gem5-users mailing list -- [email protected]<mailto:[email protected]>
To unsubscribe send an email to
[email protected]<mailto:[email protected]>
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
_______________________________________________
gem5-users mailing list -- [email protected]<mailto:[email protected]>
To unsubscribe send an email to
[email protected]<mailto:[email protected]>
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
_______________________________________________
gem5-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
