This article explains how to troubleshoot:

  • Missing events that were expected to be sent or logged to the expected Output.
  • Missing Extensions in the event sent to the Output.

 If you are not receiving expected events:

  1. Ensure that SIEM Agent been started. Use the following commands to start Central Administration and SIEM Agent:

    PTPLLIB/PPLSTRMON
    PTSALIB/PSASTRMON

 Note: If you are using the command prompt with F4, be sure to change the Commit parameter to *Yes.

  1. Ensure that the Output has been attached to the Event Source:
    1. From the Main Menu, choose 3 to open Work with Outputs.
    2. Ensure the Output exists and is active. If it is not active, use option 6 to activate.
      Note: Correct Output settings will be confirmed later in this procedure if required.
    3. Return to the Main Menu and choose 1, then 2 for the Event Source.
    4. Press F8 to ensure the correct Output is attached. If not, press F6 to add it.
  2. Be sure to confirm all changes.
  3. Ensure the Active flag is set to 1 for the Event Source, Event Description, and Subtype:
    1. From the Main Menu, choose 1.
    2. Choose 2 for the Event Source.
    3. Set Active to 1 (if it is not already) and press Enter.
    4. Choose 8 for the Event Source to open Work with Event Descriptions.
    5. Use 6 to toggle inactive events from 0 (inactive) to 1 (Active).
    6. Subtypes, if they exist, must be active as well in order to be sent to the Output. Choose 8 for the Event Description (Journal Code/Entry Type pair) to open Work with Event Subtypes. Use 6 to toggle inactive subtypes (0) to Active (1).
  4. Review Rules to confirm the expected Output is configured:
    1. From the Main Menu, choose 1, then 9 for the Event Source to show the Event Descriptions.
    2. Choose 9 for the event. If Rules are attached:
      1. Use 8 to confirm the Conditions of the Rule are correct.
      2. Use 2 to confirm the Rule settings are correct. If Rule Output is set to “None” the Rule is preventing the event from being sent to the Output.
      3. Use F8 to open Work with Attached Outputs, where you can verify the Output attached is the correct one and set to Active (1).
    3. Return to Work with Event Descriptions. If the event is a Subtype, use 8 to show the Event Subtypes for the Event Description. Go through the same steps as above for the Rules attached to the Event Description.
  1. If you made any changes, press Enter to confirm them and retest.

 If at this point, the issue has not been resolved, proceed to that section below which matches your type of Output.

For Output Intended for a SIEM (Output Type *NETWORK)

  • Determine the following information:
    1. Address (IP or name) of SIEM. (Note: Version 4.0 does not support the use of the FQDN.)
    2. Protocols the SIEM accepts (UDP, TCP, TLS).
    3. Port the SIEM accepts messages at.
  • Check firewall settings:
    • Does a firewall control the network traffic between the local system and the SIEM? If yes, ensure the firewall allows the events to be received by the SIEM. The protocol and (destination) port to open in the firewall must match those configured on the Output.
  • Ensure the correct address has been specified on the Location field of the Output:
    1. If a numerical IP address specified, double-check that the IP address is correct.
    2. If a name is specified, use a PING from that IBM i to check that the name is resolved correctly. (The PING may not be replied to, but the name must be resolved to an IP address.)
  • Ensure that the SIEM is configured to accept messages:
    1. From this IBM i (source address).
    2. Using the protocol configured on the Output.
  • Using the port configured on the Output.
  • Local-copy check:
    1. Add an Output of type *STREAM.
    2. Ensure that the directory in which the Stream File should reside already exists.
    3. Ensure that user profile PTUSER has *CHANGE authority to the directory.
    4. Add the new Output to the Default Output of Event Source.
    5. Confirm changes.
    6. Provoke the event.
    7. Observe if a message reflecting the new message/event is added to the local log:
      • If yes: Issue is likely to be firewall, network, SIEM, or mismatch between SIEM settings and configured Output pointing at the SIEM.
      • If no: Issue is more likely to be in the Event Source, Event Description, or Rules.

For Output Intended to Be Sent to a Stream File (*STREAM)

  1. Ensure that the directory in which the Stream File should reside already exists.
  2. Ensure that user profile PTUSER has *W(RITE) authority to the directory.

 For Output Intended to be Sent to a Message Queue (*MSGQ)

  1. Ensure that the message queue already exists.
  2. Ensure that user profile PTUSER has *CHANGE authority to the message queue.
  3. Ensure that the message queue is not both full and set to not *WRAP.

If Events Are Received but Extensions are Missing

When output does not contain Extensions that you defined, the Output can be configured to use the legacy *CEF or SYSLOG Message Type. This message type does not allow Extensions to be included in the message. Use an output with the *MODERN Message Type instead.

Resolving with SIEM Agent’s Tracing Feature

If the above steps did not resolve the issue:

  1. Enable tracing:
    ADDLIBLE PTSALIB
    PSATRCSIEM MONITOR(*MAIN) TRACE(*START) PATH('/tmp/PTSA-trace-main') 
    PSATRCSIEM MONITOR(name of Event Source) TRACE(*START) TYPE(*SOURCE) PATH('/tmp/PTSA-trace-source01')
    PSATRCSIEM MONITOR(name of Output) TRACE(*START) TYPE(*OUTPUT) PATH('/tmp/PTSA-trace-output01')  
  2. Recreate the issue:
    • For example, if you are trying to forward or log audit journal events, cause the audit journal entry to be deposited (created). To provoke a TAD (System Values Modification) event, for example, modify an object’s audit settings.
  3. End tracing:
    PSATRCSIEM MONITOR(*MAIN) TRACE(*STOP)
    PSATRCSIEM MONITOR(name of Event Source) TRACE(*STOP) TYPE(*SOURCE)
    PSATRCSIEM MONITOR(name of Output) TRACE(*STOP) TYPE(*OUTPUT)
  4. Transfer the trace files to your PC:
    1. Copy the following files to your PC. You can transfer them using any method available, such as FTP or the file server.
      1. /tmp/PTSA-trace-main
      2. /tmp/PTSA-trace-source01
      3. /tmp/PTSA-trace-output01
  1. Create a support case through the Community Portal. Provide the following information:
    • Description of the issue.
    • Screenshots of the configuration of the Event Source, Event Description, Subtype (if one exists), Rules attached to the Event Description, Rules attached to the Subtype.
    • The trace files.
    • Provide the trace files created and transferred in the preceding steps as attachments to the support case.

Checking for UDP Traffic

If Events are configured to be sent to an Output (SIEM) via the UDP protocol, but are not received, the following steps enable you to check if the Events are sent out.

  1. Provoke the Event.
  2. Run the command NETSTAT.
  3. Select option 3, Work with IPv4 connection status.
  4. Look for a connection with the attributes below. Note that you may need to press F11 to change the columns displayed, or to display details for a connection.
    Local host name . . . . . . . . . . . :
    Local internet address  . . . . . . :     *
    Local port  . . . . . . . . . . . . :     (portnumber is 1024 or higher) = high and random)  
    Associated user profile . . . . . . . :   PTUSER

    Bytes in  . . . . . . . . . . . . . . :   0
    Datagrams in  . . . . . . . . . . . :     0                                          

If you display the job for this connection, the job should be named PSAMGRMON.

If SIEM Agent is sending out UDP traffic, you will see the “Bytes out” and “Datagrams out” counts for the connection increase with each Event that it sends out.

Note: The connection only exists after at least one event has been sent out by SIEM Agent.

To see which server is sending data:

  1. "Display jobs" for the connection.
  2. Note the job's job number name.
  3. Submit WRKACTJOB JOB(PSAMGRMON).
  4. Press F11 twice so you see the job number.
  5. Identify the job/s by the job number.
  6. Then press F11 again to display the job function.
  7. The job function, minus the "USR-" prefix, will show you the name of the Output.

If data is being sent out by the connection, but not received by the SIEM, please check the items above (IP address, port, firewall, SIEM settings).