Firmware upgrade from version 6.0.6 to 6.2.2
(Latest version is 6.2.2, N-1 is 6.2.1 –Why we upgrade 6.2.2 ? because as per Fortinet advises customers to upgrade to FortiOS 5.4.13, 5.6.11, 6.0.6 or 6.2.2 or above and found the below vulnerabilities.
“ SSL VPN VULNERABILITIES : Security vulnerabilities discussed at the BlackHat 2019 conference
At the recent Black Hat 2019 conference held in Las Vegas August 3-8, security researchers discussed their discovery of security vulnerabilities that impacted several security vendors, including Fortinet. All of the vulnerabilities impacting Fortinet were fixed in April and May of 2019. FortiOS 5.4.13*, 5.6.11, 6.0.6 or 6.2.2 are recommended.”
More Read https://kb.fortinet.com/kb/microsites/microsite.do?cmd=displayKC&docType=kc&externalId=FD46513
We have upgraded successfully from FortiOS 6.0.6 to 6.2.2 and after some time we observed the CUP utilization goes very high unexpected.
It may be the BUG for the high CPU process.
Perform below command output from the Fortinet CLI
CLI# diagnose sys top
CLI# diagnose sys top-summary
OUTPUT
# diagnose sys top-summary
[H[JRun Time: 10 days, 18 hours and 27 minutes
14U, 0N, 45S, 4I, 37WA, 0HI, 0SI, 0ST; 1008T, 493F
wad 17778 S 1.4 0.6
wad 4724 D 0.9 0.2
wad 4725 D 0.9 0.2
newcli 4669 R 0.4 0.6
httpsd 12792 S 0.0 3.1
cmdbsvr 108 S 0.0 2.6
forticron 155 S 0.0 2.5
pyfcgid 12455 S 0.0 2.3
httpsd 5924 S 0.0 2.3
cw_acd 181 S 0.0 1.8
httpsd 148 S 0.0 1.7
miglogd 227 S 0.0 1.7
updated 169 S 0.0 1.6
miglogd 145 S 0.0 1.6
pyfcgid 12470 S 0.0 1.5
pyfcgid 12471 S 0.0 1.5
pyfcgid 12469 S 0.0 1.5
fgfmd 180 S 0.0 1.3
newcli 4483 S 0.0 1.2
initXXXXXXXXXXX 1 S 0.0 1.1
# diagnose sys top-summary
[H[J CPU [||||||||||||||||||||||||||||||||||||||||] 100.0%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=2 sleeping=108 disk sleep=1)
PID RSS ^CPU% MEM% FDS TIME+ NAME
- 22828 6M 0.0 0.6 30 00:05.55 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
145 17M 0.0 1.8 57 37:03.36 miglogd [x2]
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
152 5M 0.0 0.6 11 37:17.80 merged_daemons
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
164 6M 0.0 0.6 41 07:49.53 proxyd [x2]
165 7M 0.0 0.7 21 02:21.30 voipd
[H[J CPU [||||||||||||||||||||||||||||||||||||||| ] 97.9%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=1 sleeping=110) PID RSS ^CPU% MEM% FDS TIME+ NAME - 174 16M 14.2 1.7 13 00:00.65 sshd [x4]
17778 8M 1.7 0.9 71 28:59.29 wad [x6]
145 17M 0.8 1.8 57 37:03.37 miglogd [x2]
152 5M 0.8 0.6 11 37:17.90 merged_daemons
22828 6M 0.0 0.6 30 00:05.55 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
[H[J CPU [|||||||||||||||||||||||||||||||||||||| ] 96.6%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=2 sleeping=108 disk sleep=1) PID RSS ^CPU% MEM% FDS TIME+ NAME - 174 16M 13.3 1.7 13 00:00.81 sshd [x4]
17778 8M 0.8 0.9 71 28:59.30 wad [x6]
22828 6M 0.0 0.6 30 00:05.55 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
145 17M 0.0 1.8 57 37:03.37 miglogd [x2]
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
152 5M 0.0 0.6 11 37:17.90 merged_daemons
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
[H[J CPU [|||||||||||||||||||||||||||||||||||||| ] 95.4%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=1 sleeping=109 disk sleep=1) PID RSS ^CPU% MEM% FDS TIME+ NAME - 174 16M 13.4 1.7 13 00:00.96 sshd [x4]
17778 8M 1.7 0.9 71 28:59.32 wad [x6]
22828 6M 0.8 0.6 30 00:05.56 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
145 17M 0.0 1.8 57 37:03.37 miglogd [x2]
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
152 5M 0.0 0.6 11 37:17.90 merged_daemons
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
[H[J CPU [||||||||||||||||||||||||||||||||||||| ] 92.5%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=3 sleeping=107 disk sleep=1) PID RSS ^CPU% MEM% FDS TIME+ NAME - 174 16M 12.6 1.7 13 00:01.12 sshd [x4]
17778 8M 0.8 0.9 71 28:59.33 wad [x6]
22828 6M 0.0 0.6 30 00:05.56 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
145 17M 0.0 1.8 57 37:03.37 miglogd [x2]
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
152 5M 0.0 0.6 11 37:17.90 merged_daemons
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
[H[J CPU [||||||||||||||||||||||||||||||||||||| ] 94.6%
Mem [|||||||||||||||||||| ] 51.0% 519M/1008M
Processes: 20 (running=5 sleeping=107 disk sleep=1) PID RSS ^CPU% MEM% FDS TIME+ NAME - 174 16M 12.0 1.7 13 00:01.26 sshd [x4]
17778 8M 1.7 0.9 71 28:59.35 wad [x6]
22828 6M 0.0 0.6 30 00:05.56 scanunitd [x3]
144 5M 0.0 0.6 12 00:29.32 uploadd
145 17M 0.0 1.8 57 37:03.37 miglogd [x2]
147 5M 0.0 0.5 8 00:00.00 kmiglogd
148 37M 0.0 3.7 23 03:36.18 httpsd [x4]
150 5M 0.0 0.6 8 01:42.71 getty
151 6M 0.0 0.6 12 00:35.65 ipsmonitor
152 5M 0.0 0.6 11 37:17.90 merged_daemons
153 8M 0.0 0.9 15 00:13.70 fnbamd
154 5M 0.0 0.6 11 00:42.72 fclicense
155 25M 0.0 2.6 24 13:12.92 forticron
156 10M 0.0 1.0 17 01:18.52 forticldd
157 8M 0.0 0.8 44 00:07.82 authd [x3]
158 8M 0.0 0.8 23 00:03.44 foauthd
159 5M 0.0 0.6 14 00:45.79 clearpass
160 6M 0.0 0.6 10 00:00.44 httpclid
161 6M 0.0 0.6 11 00:00.12 fas
163 5M 0.0 0.6 10 00:02.38 fsso_ldap
noticed that couple of WAD process was in D state.
So I would suggest you to reboot the Fortigate device to recover from D state.
If still problem persist please share below out put.
get sys status
get sys per status (run this command 5 times in interval of 1 minutes)
diag sys session stat
diag hardware sysinfo memory
diag sys top 4 40 (run this command for 40 seconds)
diag sys top-summary
diagnose sys session full-stat
diag sys session stat
fnsysctl cat /proc/stat
fnsysctl cat /proc/interrupts
diag hard sys slab
fnsysctl df -k
fnsysctl ls -l /tmp
diag ips session status
diag ips memory pool
diag ips share pool
diag ips signature status
diag ips dissector status
diag ips packet status
diag test application ipsmonitor 13
diag debug report
Network and Security Solutions 
Leave a comment