Splunk & Active Directory Password Expiry

So the other day I was asked by our IT Helpdesk if I could send them an alert when a VIP user's password is close to expiring so that they could pro-actively go through the laborious process of changing the password on multiple devices (a topic for another day)

Since it took me a bit of time to get it right, I thought I would share the search I used so that it might help someone else as well.

If you spot errors (which is quite easy when trying to work with Active Directory timestamps), let me know so we can fix it for everyone.

The search uses "ldapsearch" of course, which is yet again a topic for another day, and assumes that your password expiry is set to 30 days and you want to be warned 5 days before it expires.

It looks like this:

| ldapsearch domain=DOMAIN search="&(objectCategory=user)(|(sAMAccountName="user1")(sAMAccountName="user2"))"
| eval pwd=strptime(pwdLastSet,"%Y/%m/%d %T")
| eval pwdExpires=pwd+(30*86400)
| eval pwdAge=round((now()-pwd)/86400,0)
| convert timeformat="%Y/%m/%d %H:%M:%S" ctime(pwdExpires) as pwdExpires
| table cn,description,userAccountControl,sAMAccountName,pwdLastSet,pwdExpires,pwdAge
| sort pwdExpires
| where pwdAge > 25

Installing Metasploit and Armitage on Mac OSX 10.9 Mavericks

Like most people out there, I have tried to install Metasploit and Armitage using other blog posts first and found that the process failed somewhere along the line.
This is yet another attempt to document my experience with the installation, that does borrow heavily from other sources, with a few minor tweaks.

Maybe, just maybe, this is the one that works for you too from start to finish....

Assumptions:

I assume that like me, you have a pretty fresh mac to do the installation on. This will probably fail spectacularly if you have macports installed already, or have upgraded from a previous OSX version with metasploit installed.

Metasploit Installation:

 

XCode:

1. Install Xcode from the App Store, its free
2. Now its imperative that you install the command line developer tools. On mavericks its done by running 

xcode-select --install


The Xcode command line tools installation MUST complete successfully before you continue

MacPorts:

1. Install MacPorts from http://www.macports.org/install.php
2. After installation run

sudo port selfupdate
sudo port upgrade outdated
sudo port install nmap
sudo port install wget

Ruby and Friends (Ruby Version Manager):

 

curl -#L https://get.rvm.io | bash -s stable --autolibs=3 --ruby 
 
After the installation of RVM, you often have to close the terminal and re-open it before you do the following:

rvm requirements
rvm install ruby-1.9.3-p448
rvm gemset create msf
rvm use ruby-1.9.3-p448 --default
source ~/.rvm/scripts/rvm

Ruby Gems:

gem install sqlite3 msgpack hpricot

PostgreSQL:

sudo port install postgresql93-server
gem install pg -v '0.16.0' -- --with-pg-config=/opt/local/lib/postgresql93/bin/pg_config

sudo gem install pg -v '0.16.0' -- --with-pg-config=/opt/local/lib/postgresql93/bin/pg_config

Yes I did both a normal and a sudo install, seemed it was needed on my system 

Create a database instance:

 sudo mkdir -p /opt/local/var/db/postgresql93/defaultdb
 sudo chown postgres:postgres /opt/local/var/db/postgresql93/defaultdb

Start the database server:

sudo su postgres -c '/opt/local/lib/postgresql93/bin/postgres -D /opt/local/var/db/postgresql93/defaultdb' &

To start the server automatically at boot time:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.postgresql93-server.plist

Setup your PATH:

export PATH=/opt/local/lib/postgresql93/bin:$PATH

Create the user and database:

createuser msf -h localhost

createdb metasploitdb

Install Metasploit:

 

sudo su
mkdir -p /opt/
cd /opt
git clone https://github.com/rapid7/metasploit-framework.git msf
exit
export PATH=/opt/msf:$PATH

Configure the Metasploit Database:

 

sudo echo export MSF_DATABASE_CONFIG=/opt/msf/database.yml >> ~/.bash_profile
sudo vi /opt/msf/database.yml

Paste the following into the database.yml file

production:
   adapter: postgresql
   database: metasploitdb
   username: msf
   password:
   host: 127.0.0.1
   port: 5432
   pool: 75
   timeout: 5

Install Metasploit Bundles:

cd /opt/msf/
sudo bundle install

Create symlinks for Armitage:

 

sudo ln -s /opt/msf/msf* /opt/local/bin/

Run Metasploit!

 

cd /opt/msf
sudo msfconsole

See if you can connect to the database

msf > db_connect msf@metasploitdb
[*] Rebuilding the module cache in the background...
msf > db_status
[*] postgresql connected to metasploitdb
msf > workspace
* default

Armitage Installation:

 

Download and install Armitage

 

Click here to to download Armitage for OSX.
Armitage should be installed the same way you would install any other OSX app. Open the DMG file, and drag the application to your Applications folder.

Download and install Java:

 

Apple is no longer developing its own version of Java, it is recommended to use the Oracle version of Java that can be downloaded and installed from http://www.java.com/en/download/mac_download.jsp?locale=en

You also need to install the JDK so you can run java from the command line

http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-macosx-x64.dmg

Start Armitage and MSFRPCD:

 

Starting Armitage with sudo from the command line seems to work best, and also starts MSFRPCD for you

sudo -E java -jar /Applications/Armitage.app/Contents/Java/armitage.jar


 

Installing Snorby on Ubuntu 12.04

Prerequisites:

This guide assumes you have setup a clean Ubuntu Server 12.04 image with only SSH installed
Let's get started:

# sudo su -
# apt-get update
# apt-get upgrade
# apt-get install mysql-server
# apt-get install apache2
# apt-get install libyaml-dev git-core default-jre imagemagick libmagickwand-dev wkhtmltopdf gcc g++ build-essential libssl-dev libreadline-gplv2-dev zlib1g-dev linux-headers-generic libsqlite3-dev libxslt1-dev libxml2-dev libmysqlclient-dev libmysql++-dev apache2-prefork-dev libcurl4-openssl-dev -y

Install Ruby & Friends

# apt-get install ruby1.9.3 ruby-text-format
# gem install bundler 
# gem install rails
# gem install rake --version=0.9.2

Install Snorby

# cd /var/www/
# git clone http://github.com/Snorby/snorby.git

# cd /var/www/snorby/config/ 
# cp database.yml.example database.yml
# cp snorby_config.yml.example snorby_config.yml
# sed -i s/"\/usr\/local\/bin\/wkhtmltopdf"/"\/usr\/bin\/wkhtmltopdf"/g /var/www/snorby/config/snorby_config.yml

Configure snorby database username and password

# vi database.yml

Edit the settings to match your MySQL installation

# cd /var/www/snorby/
# bundle install --deployment
# rake snorby:setup

Setup Apache

# gem install passenger
# passenger-install-apache2-module

Copy the lines provided at the end of the installation script

 # vi /etc/apache2/apache2.conf

The lines should look something like this at the end of the file

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.
   LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
   PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.18
   PassengerRuby /usr/bin/ruby1.9.1

# Include generic snippets of statements
Include conf.d/

# Include the virtual host configurations:
Include sites-enabled/


Now add the website as follows:

# vi /etc/apache2/sites-available/snorby

Add the following lines (change the ServerAdmin and Servername to something of your choice)

<VirtualHost *:80>
        ServerAdmin admin@localhost
        ServerName snorby.localhost
        DocumentRoot /var/www/snorby/public

        <Directory "/var/www/snorby/public">
                AllowOverride all
                Order deny,allow
                Allow from all
                Options -MultiViews
        </Directory>
</VirtualHost>

Enable the new website and disable the default site

# a2dissite default
# a2ensite snorby
# service apache2 reload

Browse to the new website and login with the following default credentials:

Username: snorby@snorby.org
Password: snorby

Be sure to change the default credentials after your first login

You should see a page that looks something like this

Be sure to start the Snorby worker in the interface by clicking on Administration -> Worker Options -> Start Worker

Mysql and Snort Sensor Setup

Unfortunately our work is not done. Depending on your setup, you still need to get the snort sensors to log to your Snorby server. In my case I have seperate, remote snort sensors so to get them to log to snorby I have to do the following:

We first need to get MySQL to listen on for remote connections, as this is turned off by default these days

# vi /etc/mysql/my.cnf

Uncomment the bind-address line like so

#bind-address           = 127.0.0.1

Restart MySQL

#service mysql restart

Now we need to create users for the various snort sensors like so. The below assumes your snort sensor is running on 10.0.0.10, so adjust accordingly

# mysql
mysql> CREATE USER 'snort'@'10.0.0.10' IDENTIFIED BY 'snortsnort';
mysql> GRANT ALL PRIVILEGES ON snorby.*TO 'snort'@'10.0.0.10';
mysql> FLUSH PRIVILEGES;

Now on your snort sensor, either in your snort.conf file or barnyard.conf file, whichever you are using, you need to have a line that looks something like this (again, change to match your situation)

output database: log, mysql, user=snort password=snortsnort dbname=snorby host=10.0.0.11

Troubleshooting

After installation my Snorby worker did not want to start through the web interface, so I started it manually like so

# rails c production
Loading production environment (Rails 3.1.0)
irb(main):001:0> Snorby::Worker.stop
=> ""
irb(main):002:0> Snorby::Jobs.clear_cache
=> nil
irb(main):003:0> Snorby::Worker.start
=> ""
irb(main):004:0> exit

Fine Tuning Snorby

Once the events start rolling in, you will probably see a lot of events that are classified as High that you would to change to Medium or Low

I normally do this by getting the signature id from the interface first and then I change the severity in the database directly like so

# mysql snorby

mysql> select * from signature where sig_sid = "100000230";
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
| sig_id | sig_class_id | sig_name                                          | sig_priority | sig_rev | sig_sid   | sig_gid | events_count |
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
|      6 |            1 | GPL CHAT MISC Jabber/Google Talk Outgoing Traffic |            1 |       2 | 100000230 |       1 |         2667 |
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
1 row in set (0.00 sec)

mysql> update signature set sig_priority = 3 where sig_sid = "100000230";
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

Feature Request! It would be nice to be able to do this directly from the interface

Install Metasploit and Armitage on Mac OSX Mountain Lion

Prerequisites:

• Install the latest version of XCode from the App store
• Install the command line tools in Xcode (Xcode -> Preferences -> Downloads)

 

• Install MacPorts from http://www.macports.org/install.php
• Increase the shared memory settings in Mac OSX as these are needed by PostgreSQL later

# sudo vi /etc/sysctl.conf

On a MacBook Pro with 2GB of RAM, my sysctl.conf contains:

kern.sysv.shmmax=1610612736
kern.sysv.shmall=393216
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.maxprocperuid=512
kern.maxproc=2048

Reboot your machine before proceeding for the above settings to take effect

PostgreSQL Installation:

• Download and install PostgreSQL from http://www.enterprisedb.com/products-services-training/pgdownload#osx
• Setup your root PostgreSQL password during installation.
• Launch the newly installed "PGAdmin III" application.
• Connect (double click) on the local PostgreSQL database and enter your root password when prompted.
• Under the PostgreSQL drop down, right click on “Login Roles”, and select “New Login Role”
• Set the role name to msfuser.
• Click on the definition tab, and set the password as msfpassword.
• Click OK to continue.
• Next, right click on the databases list, and select “New Database”.
• Set the name to metasploitdb, and set the owner to msfuser.
• Press OK, and we’re done. You can close PGAdmin. 

Ruby and RubyGems installation: 

# sudo port selfupdate
# sudo port install ruby19 +nosuffix
# sudo env ARCHFLAGS='-arch x86_64' gem install pg -- --with-opt-include=/Library/PostgreSQL/9.1/include/ --with-opt-lib=/Library/PostgreSQL/9.1/lib/
# sudo port install msgpack
# sudo gem install msgpack
# export PATH=/opt/msf3:$PATH

Metasploit 4 Installation

# sudo svn co https://www.metasploit.com/svn/framework3/trunk/ /opt/msf3/
# sudo ln -s /opt/msf3/msf* /opt/local/bin
 

Configure the Metasploit Database

# sudo mkdir /opt/local/config
# sudo vi /opt/local/config/database.yml

The config file should look something like this

production:
  adapter: "postgresql"
  database: "metasploitdb"
  username: "msfuser"
  password: "msfpassword"
  port: 5432
  host: "localhost"
  pool: 256
  timeout: 5

Add the location of the database file to your environment

# export MSF_DATABASE_CONFIG=/opt/local/config/database.yml

Create pidof

You need to create a pidof file as the Armitage team server uses it to check if msfrpcd is running already. Since pidof does not exist on MacOSX natively, we will have to hack one together

# sudo vi /usr/local/bin/pidof

Add the following in the file:

#!/bin/sh
ps axc|awk "{if (\$5==\"$1\") print \$1}"|tr '\n' ' '
echo

Make the file executable

# sudo chmod 755 /usr/local/bin/pidof

Run Metasploit and Armitage

Start the Armitage team server

# cd /opt/msf3/data/armitage
# sudo -E ./teamserver <ip address> <password>

Note the -E option for sudo to keep the environment variables

Troubleshooting  

Q: When trying to start the teamserver as root I get the following error:
[*] MSGRPC starting on 127.0.0.1:55553 (SSL):Msg...
[*] MSGRPC ready at Thu Dec 20 12:03:26 +0200 2012.
/opt/msf3/lib/fastlib.rb:374:in `fastlib_original_require': no such file to load -- msgpack (LoadError)
    from /opt/msf3/lib/fastlib.rb:374:in `gem_original_require'
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems
 
A: As root, I ran "gem install msgpack" again and the error went away

Q: I get the following error when trying to start the teamserver

Warning: Uncaught exception: java.lang.RuntimeException: I can not find a database.yml file. I *really* need it.
Try setting MSF_DATABASE_CONFIG to a file that exists. at preferences.sl:121

A: Run "export MSF_DATABASE_CONFIG=/opt/local/config/database.yml" and make sure to use the -E flag for sudo

Installing Snort with ETPRO rules on Ubuntu 12.04

 Prerequisites

You first need to get a base image up and running which we won't cover here.
This guide also assumes that you are already spanning relevant traffic to eth1 of your snort box

Now you need to install Snort. Generally speaking I like using the source repositories as its easier to maintain and update. So lets get going...

# sudo su -
# apt-get update
# apt-get upgrade
# apt-get install snort-mysql

When prompted for the default home range, I normally use 10.0.0.0/8, yours might be different but most internal networks are in the 10.x.x.x range.

When asked whether a snort database should be created, choose "Yes" as we will be logging to Snorby that will be running on a separate server but that we will configure later

I always install swatch as I use it to monitor the log files for error messages

# apt-get install swatch

Now we need to install oinkmaster that will keep snort updated

# apt-get install oinkmaster

Now we need to configure oinkmaster to use our ET PRO rules instead of the default ones

First check what version of Snort you are running

# snort -V

In my case the version is 2.9.2 as can be seen below


   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.2 IPv6 GRE (Build 78)
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2011 Sourcefire, Inc., et al.
           Using libpcap version 1.1.1
           Using PCRE version: 8.12 2011-01-15
           Using ZLIB version: 1.2.3.4

Now edit the oinkmaster.conf file and add the ET Pro line like so

# vi /etc/oinkmaster.conf

Add the following line to the file

url = http://rules.emergingthreatspro.com/<etpro code goes here>/snort-2.9.2/etpro.rules.tar.gz

Next up you need to edit snort.conf to reflect your preferences. Below are the settings that I normally change, just search for them in the snort.conf file

# vi /etc/snort/snort.conf

Here are my changes:

# syslog
output alert_syslog: LOG_LOCAL7 LOG_ALERT

# pcap
#output log_tcpdump: tcpdump.log

Just before "Step #8:", I add the following

# Include the ETPRO rules
include $RULE_PATH/etpro.conf

Configure Swatch

Edit the swatch.conf file

# vi /etc/swatch.conf

Add the following line

watchfor /(ERROR)/
        echo=red
        mail addresses=<put your email address here>,subject=Snort FATAL Error

Starting Snort

Now I prefer starting and stopping snort via a cron script as can control exactly what is going on. So create the following script:

# vi /etc/cron.daily/5.snort

Add the following lines to the file:

#!/bin/sh -e
test -f /usr/sbin/snort || exit 0
PATH=/bin:/usr/bin:/sbin:/usr/sbin

# Kill Snort
kill -INT `cat /etc/snort/snort_eth1.pid` || true

# Kill Swatch
kill -INT `cat /etc/swatch.pid` || true

sleep 10

# Delete log files to keep the disks clean
/bin/rm /var/log/snort/unified* || true
/bin/rm /var/log/snort/snort* || true

# Restart SYSLOG so everything is nice and clean
service rsyslog restart || true

#Run Oinkmaster
/usr/sbin/oinkmaster -C /etc/oinkmaster.conf -o /etc/snort/rules

# Start swatch
/usr/bin/swatch --config-file=/etc/swatch.conf --tail-file=/var/log/syslog --daemon --pid-file /etc/swatch.pid

sleep 10

# Start SNORT
/usr/sbin/snort -x -c /etc/snort/snort.conf --pid-path /etc/snort -i eth1 2>>/var/log/syslog &

exit 0

Another reason I start snort like this is also to be able to catch error messages with swatch and email it to myself which will come in very handy, trust me...

Now we run the cron file to start everything up

# /etc/cron.daily/5snort

If all goes according to plan, snort should start up and you can see the snort alerts by running

# tail -f /var/log/syslog

But things rarely go acccording to plan so you probably need to fix a few things....

Troubleshooting

Since I use swatch, I get the following error popping up on my terminal which shows something went wrong when I try and start up snort

ERROR: /etc/snort/rules/policy.rules(298) !any is not allowed: ![$DNS_SERVERS,$SMTP_SERVERS].

This means there is an unsupported option in the policy.rules file. Oinkmaster to the rescue since it can be used to modify rules as its downloaded

At the bottom of the /etc/oinkmaster.conf file, add the following line

disablesid 2003195

The above line disables the rule that is giving us trouble.

Now run the cron file again and deal with the next issue in a similar fashion

Open Source Network Forensics and Zero-Day Malware

Although I work at a large enterprise with enough budget for the best security tech money can buy, I still have a passion for open source tools and a deep admiration for the people that freely give up their time and energy to build things of value and then share it with others.

We have successfully implemented a combination of open source tools, particularly in the field of network forensics, to detect and alert on malware traffic that was missed by the expensive tech, although they of course do their fair share of detecting and blocking as well.

It's obvious that now, more than ever, a defense in depth strategy is needed to combat what is in my opinion the #1 threat organizations face today - malware.

So why yet another blog...

Inspired by the name of one of the blogs I read regularly, The Day Before Zero, I've decided to put words into action and start a blog where we (me and others in my team) can share our experiences and learnings from the trenches of IT Security.

Because that is what we do, we defend the organizations we work for on a daily basis against the bad guys out there. So hopefully this blog will help in bolstering your defenses and keeping your company out of the headlines.