Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This morning I'm receiving complaints from several end users that saying their whole department network are slow and have intermittent.

Therefore I've checked our firewall to see whether if something goes wrong with the device.From my observation in the FortiGate dashboard status, the CPU resources is very high (99 percent). My first assumption is to clear the log since in the alert log the Fortigate log mention that it is already 90% full.Based on my understanding,the log can be cleared by restarting the firewall.

After restarting the firewall the network seems okay again but then after several minutes it went up again.The condition still persist until now.

Can someone show me where else I can check to fix this issue? I've really appreciate any help that I can get here.Thanks.

Edit:

diag sys top command

share|improve this question
    
Any unusual network activity at the machines behind the firewall? Install e.g. wireshark on one machine and monitor the network when it goes "slow". Unfortunately, you don't have a clean baseline to compare. Download the logs from the firewall and analyze them. Maybe you are being the subject of some kind of attack, or a worm went wild in your network (update and run all antivirus and other malware deletion software at hand, including clamav). –  vonbrand Mar 11 '13 at 3:40
    
so far from my end, nothing unusual network activity behind the firewall.but there is possibility for the attack as you have mention. I've also tried to disable some of the service in order to reduce the CPU resources but I'm unable to stop the service (for example: ips,antivirus, etc..).It showing me this error 'CFG_CMDBAPI_ERR' –  user119720 Mar 11 '13 at 4:29
    
Could you run #diag sys top on it and come back with your results? Your issues could be due to the summary reports daemon (sumreportsd), or some other process taking too much CPU time, but we need to see which one first to be able to offer suggestions. To get rid of 'CFG_CMDBAPI_ERR' error restart it with #diag test app ipsmonitor 99. –  TildalWave Mar 11 '13 at 4:34
    
@TildalWave I've already put the output for #diag sys top command above.I want to test #diag test app ipsmonitor 99 command but there are many user currently using the internet.So I cannot restart the firewall. –  user119720 Mar 11 '13 at 5:03
    
OK, nothing too apparent in the list there, but that would be the current usage. You'll have to run diagnostics for a bit longer on it, and to do that follow instructions here under 'the Fortigate CPU or Memory at 100%'. Then please replace that image with the new diagnostics (or add to it), so we see what's eating CPU in a longer time frame. –  TildalWave Mar 11 '13 at 5:15

2 Answers 2

I've already resolved this issues.For the sake to help others who might facing the same problem, I'm writing this down.

When tracing back to our firewall log, I've found out that there are several log indicating VPN SSL error. After comparing the log with the time of the occurrence of network intermittent as well as the high CPU percentage resources,I've conclude that the VPN service for this firewall is the root for all these problem.

Then after disabling the VPN service,the network went back normal until today.Something within the VPN service is eating up the CPU resources greedily. Regardless,we already plan to buy a new Fortinate firewall model to accommodate the increasing number of users in our workplace.

share|improve this answer

You can hit the letter S for the highest CPU and you can restart that application by killing the PID - Process ID diag sys kill 11 903 (9 or 11 will kill the PID and restart it 9 is safer 11 keeps no track of you killing it in logs so try 9 first then 11). usually the CPU will go back down. if it happens frequently then i would suggest installing a new Forti IOS image on the box no need to replace the box since these are software and not hardware related. for the most part.


CLI# diagnose sys top

Run Time: 13 days, 13 hours and 58 minutes 0U, 0S, 98I; 123T, 25F, 32KF newcli 903 R 0.5 5.5 sshd 901 S 0.5 4.0

Where the codes displayed on the second output line mean the following:

  • U is % of user space applications using CPU. In the example, 0U means 0% of the user space applications are using CPU.
  • S is % of system processes (or kernel processes) using CPU. In the example, 0S means 0% of the system processes are using the CPU.
  • I is % of idle CPU. In the example, 98I means the CPU is 98% idle.
  • T is the total FortiOS system memory in Mb. In the example, 123T means there are 123 Mb of system memory.
  • F is free memory in Mb. In the example, 25F means there is 25 Mb of free memory.
  • KF is the total shared memory pages used. In the example, 32KF means the system is using 32 shared memory pages.

Each additional line of the command output displays information for each of the processes running on the FortiGate unit. For example, the third line of the output is:

newcli 903 R 0.5 5.5

Where:

  • newcli is the process name. Other process names can include ipsengine, sshd, cmdbsrv, httpsd, scanunitd, and miglogd.
  • 903 is the process ID. The process ID can be any number.
  • R is the state that the process is running in. The process state can be: o R running. o S sleep. o Z zombie. o D disk sleep.
  • 0.5 is the amount of CPU that the process is using. CPU usage can range from 0.0 for a process that is sleeping to higher values for a process that is taking a lot of CPU time.
  • 5.5 is the amount of memory that the process is using. Memory usage can range from 0.1 to 5.5 and higher.

Interactive diagnose sys top commands

You can enter the following single-key commands when diagnose sys top is running.

  • Press q to quit.
  • Press c to sort the processes by the amount of CPU that the processes are using.
  • Press m to sort the processes by the amount of memory that the processes are using.

Stopping running processes

You can use the following command to stop running processes:

diagnose sys kill

Where:

  • signal can be any number but 11 is preferred because this signal sends output to the crashlog which can be used by Fortinet Support to troubleshoot problems.
  • process id is the process ID listed by the diagnose sys top command.

For example, to stop the process with process ID 903, enter the following command:

diagnose sys kill 11 903

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.