Forward logs to a remote server

This section covers the procedure to resolve issues with forwarding logs to a remote server.

To resolve the issue, the steps covered in this section must be performed on each controller from where the logs are forwarded:

• each network controller/interconnect

• each compute node

• each storage controller (both active and passive)

To forward logs to a remote server, follow these steps:

Note

Perform these steps on each compute node and storage controller in the stack.

1. Estable an SSH session with the compute node or storage controller.

   The user is manager and the password is the one chosen when the hardware was registered.

2. Execute the following command and re-enter the manager password to gain root access.

   sudo su

   If you are unable to gain root access, contact Lenovo Support.

3. Edit the rsyslog configuration file by executing the following command:

   vi /etc/rsyslog.conf and then type i

4. To forward to a remote server, add the following lines to the bottom of the file:

   # start forwarding rule
   $ActionQueueType LinkedList # use asynchronous processing
   $ActionQueueFileName srvrfwd # set file name, also enables disk mode
   $ActionResumeRetryCount -1 # infinite retries on insert failure
   $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
   *.* @@server:port
   

5. Execute the following command to save the changes and close the file:

   :wq

6. Replace server with the name or IP address of the remote server where the logs are being forwarded to.

   :port is optional and can be deleted, but if needed, you can replace it with the server port receiving the logs.

7. To forward to multiple remote servers, add the following lines to the bottom of the file:

   # start forwarding rule 1
   $ActionQueueType LinkedList # use asynchronous processing
   $ActionQueueFileName srvrfwd1 # set file name, also enables disk mode
   $ActionResumeRetryCount -1 # infinite retries on insert failure
   $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
   *.* @@server1:port
   # end forwarding rule 1 
   
   # start forwarding rule 2
   $ActionQueueType LinkedList # use asynchronous processing
   $ActionQueueFileName srvrfwd2 # set file name, also enables disk mode
   $ActionResumeRetryCount -1 # infinite retries on insert failure
   $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
   *.* @@server2:port
   # end forwarding rule 2
   

8. Execute the following command to save the changes and close the file:

   :wq

9. Replace server1 and server2 with the names or IP addresses of the servers where the logs are being forwarded to.

   • :port is optional and can be deleted, but if needed, you can replace it with the server port receiving the logs.

   • The $ActionQueueFileName must be unique in thersyslog.conf file, which is why there is a different name in each forwarding rule.

10. Execute the following command to restart rsyslog:

    service rsyslog restart