Splunk EventGen — Quick Tutorial

Whenever I’ve needed to generate data for Splunk, I’d write a small python script and monitor the output of the file. During my last Splunk engagement, $CLIENT requested that I tweak their existing EventGen app to update the format of the data being generated. I threw this little project together to refresh myself since it’d had been a while since I’ve used the app.

Eventgen is a Splunk app that generates data that can be useful for building dashboards, testing, and a lot more. When enabled and properly configured, the app will generate data based on the settings of the eventgen.conf files under each app installed located in opt/splunk/etc/apps.

Generating data, such as vulnerability scanning results, can becomes super easy when you can just install the Tenable Add-On and within seconds, have nicely formatted data populating in Splunk using this app’s default eventgen.conf file. The Add-on also includes a directory called samples that is used as a starting template for EventGen to use.

In this article, I’ll show you how to get a quick EC2 instance running EventGen. Then, I’ll make a modification to that important file (eventgen.conf), restart, and show you the difference.

First, I started by configuring an instance to launch using the Amazon Linux 2 AMI.

I bootstrapped some commands (see below) to set up an EC2 instance with Splunk installed and EventGen configured.

NOTE: Whenever you are bootstrapping, don’t forget the #!/bin/bash so the shell knows which interpreter to use.

These commands are essentially downloading and configuring Splunk with EventGen running. Then, I’m coping over the sample app files from my Github to began populating events before I even log in.

#!/bin/bash
sudo yum -y update
sudo yum -y install wget
sudo yum -y install git
git clone https://github.com/pyraven/splunk-eventgen-guide /tmp/splunk-eventgen-guide
wget -O /tmp/splunk-7.2.3–06d57c595b80-linux-2.6-x86_64.rpm ‘https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.3&product=splunk&filename=splunk-7.2.3-06d57c595b80-linux-2.6-x86_64.rpm&wget=true'
sudo rpm -i /tmp/splunk-7.2.3–06d57c595b80-linux-2.6-x86_64.rpm
sudo cp -r /tmp/splunk-eventgen-guide/tutorial/ /opt/splunk/etc/apps/
sudo tar -xvf /tmp/splunk-eventgen-guide/eventgen_632.tgz -C /opt/splunk/etc/apps/
sudo sed -i ‘s/disabled = true/disabled = false/g’ /opt/splunk/etc/apps/SA-Eventgen/default/inputs.conf
/opt/splunk/bin/splunk start — accept-license — answer-yes — no-prompt — seed-passwd sPluNkP@ssW0rD
/opt/splunk/bin/splunk stop
sudo systemctl stop Splunkd
sudo cd /tmp/
sudo chown -Rh splunk:splunk /opt/splunk/
sudo /opt/splunk/bin/splunk enable boot-start -user splunk
sudo su
echo ‘splunk ALL=(root) NOPASSWD: /usr/bin/systemctl restart Splunkd.service’ >> /etc/sudoers
echo ‘splunk ALL=(root) NOPASSWD: /usr/bin/systemctl stop Splunkd.service’ >> /etc/sudoers
echo ‘splunk ALL=(root) NOPASSWD: /usr/bin/systemctl start Splunkd.service’ >> /etc/sudoers
echo ‘splunk ALL=(root) NOPASSWD: /usr/bin/systemctl status Splunkd.service’ >> /etc/sudoers
reboot

For storage, I increased the size from 8GB to 20GB. Believe me, if you skip this part, Splunk will complain at some point.

When setting up the Security Group, I made sure to include Splunk’s default web server port (8000) and SSH (Port 22) for inbound traffic based on my source IP. You will need to adjust the security group based on your location.

The whole set up takes about 5~10 minutes before you can log in and see the data being generated.

~ 10 minutes laterish ~

I logged into the Splunk web (http://<your-ec2-public-ip>:8000) and saw the new data was being successfully generated. Woot!

Under the search app, search for sourcetype=”sample_data”

Now I wanted to modify the data. This requires SSHing into the instance, modifying the correct eventgen.conf file, and restarting Splunk. You’ll began seeing changes shortly when you log back in and search.

I’m going to quickly break down some of components of the eventgen.conf file; not all of them since there’s a lot of customization possible. At the very top of this file is starting template filename that will set the format of the events to be generated, [tutorial.samples]. You can find this file under samples/tutorial.samples in the sample Splunk app (/opt/splunk/etc/apps/tutorial).

cat tutorial.samples
##start_time##,##host_name##,##random_number##
  • Index — is where the events will be sent
  • Interval — how often events are created, default is 60 seconds
  • Count — number of events to created per interval
[tutorial.samples]
index = main
interval = 60
count = 10
earliest = -60m
latest = now
sourcetype = sample_data

This part established the format of the timestamp.

## replace start_time
token.0.token = (##start_time##)
token.0.replacementType = timestamp
token.0.replacement = %a %b %d %H:%M:%S %Y

This first part selects a random string from list under samples/host_name.sample while the second part creates a random integer in the range of 1–1000.

## replace host_name
token.1.token = (##host_name##)
token.1.replacementType = file
token.1.replacement = $SPLUNK_HOME/etc/apps/tutorial/samples/host_name.sample
## replace random_number
token.2.token = (##random_number##)
token.2.replacementType = random
token.2.replacement = integer[1:1000]

I SSH’d into my instance and navigated to opt/splunk/etc/apps/tutorial/default directory. I updated the eventgen.conf as shown here:

I just changed the very last line

I changed the random number range to 5000–9000. After saving the file, I restarted Splunk using the permissions created during instance launch.

sudo systemctl restart Splunkd.service

I logged into Splunk web and saw that the change was successful. Not too bad eh?

Success!

If there is anything I can clear up, please let me know. Thanks for reading. Cheers fellow Splunker!