Dark Training

How to remove old versions of objects in S3 when using S3 versioning.

Sat, 23 Dec 2017 12:19:00 -0800

I wanted to show readers how they can reduce their Amazon S3 costs if they are hosting static content on Amazon Web Services S3.

One of the nice functions of S3 is that it will allow you to version a bucket. For a blog site, this is a great feature that can save you if you accidentally commit a change that causes major issues on your site. However, one draw back is that each time you commit a change, a new version is created. If your site has many large files, that can add up quickly, so how can you ensure you only keep data as long as you need it?

S3 Lifecycle rules

In the S3 console for your bucket, you can enable a lifecycle rule that will expire out old versions of an object after a given window of time.

First select your bucket in the S3 console and then choose the “Management” tab.

Next select the Lifecycle Button and then press the “+ Add lifecycle rule” below it.

Enter a name for the rule and you can skip the prefix option if you want this to impact the entire bucket, then press next.

We can skip the 2nd screen which is “Transistions” because we dont want to move the old versions to S3 infrequent access or glacier, we want to delete them, so press next again.

In my example, we are going to remove all old versions 30 days after they were last versioned, and we are going to select the permanently delete option. Note we also use the clean up options that are on the lower screen as well.

That’s it, simply select next, review and then complete the rule.

NameCheaps private email MX settings for AWS Route53

Sun, 03 Jan 2016 06:19:00 -0800

When you are moving your Website from Namecheap to AWS, one the functions you may want to keep is the email hosting from Namecheap. However, after you move your DNS to point route53, you’ll also need to make a MX (mail exchange) record for Route53 in order to have this work.

I was a little surprised to see that I did not find a site that basically anwserd “What do I need to type into the route53 MX value”.

Name cheap provides a block of text that looks like the image below but they don’t provide much else, and why would they, it’s not their service anymore.

This domain name is using third party DNS, please contact your DNS provider to add the following records: 
.mail(YOUR-DOMAIN).com IN
MX mx1.privateemail.com. 
TXT v=spf1 include:spf.privateemail.com ~all 
MX mx2.privateemail.com.
.autodiscover(YOUR-DOMAIN).com IN CNAME privateemail.com.
.autoconfig(YOUR-DOMAIN).com IN CNAME privateemail.com.
._autodiscover._tcp(YOUR-DOMAIN).com IN SRV 0 443 privateemail.com.

In route53, create a new record set as type “MX”, then in the values simply paste:

 10 mx1.privateemail.com.
 20 mx2.privateemail.com.

Now, if you want your subdomain mail.<your-domain>.com to redirect to their private mail server, create a CNAME and alias ‘mail.<your-domain>.com’ to point to ‘‘priavateemail.com’’.

New face lift for the site!

Sun, 27 Dec 2015 01:00:00 -0800

Well, after a good long while I’ve taken the plunge and moved from my old dynamic site to a static content driven site.

I did this for a few reasons but one of the more important was security. The old site relyed on php and a RDBMS in the background to host the content and no matter how many safegards I put up, it always made me un-easy.

Now with the new site, the page loads should be faster and the navigation should be slightly easier. I’ve also removed the comments for a little while as well. I greatly enjoyed hearing from viewers but bots were constantly trying to post and it was getting tiring to filter them out.

Windows 2008 R2 stuck in recovery mode

Wed, 05 Nov 2014 15:31:43 -0800

Recently I came across a Windows Server 2008 R2 stuck in recovery mode after reboot.

Booting with a Windows 2008 R2 recovery DVD, the following steps allowed the machine to boot normally.

Put the Windows Server 2008 R2 installation disc in the disc drive, and then start the computer.
Press any key when the message indicating "Press any key to boot from CD or DVD …". appears.
Select a language, time, currency, and a keyboard or another input method. Then click Next.
Click Repair your computer.
Click the operating system that you want to repair, and then click Next.
In the System Recovery Options dialog box, click Command Prompt.
Type Bootrec /RebuildBcd, and then press ENTER.

If that does not work.

dism.exe /image:C:\ /cleanup-image /revertpendingactions

"sfc /scannow /Offbootdir=c:\ /Offwindir=c:\windows

Mavericks install "This copy of the Install OS X Mavericks application can't be verified"

Fri, 14 Mar 2014 04:49:58 -0700

Recently I ran across and issue where we could not install a fresh copy of OS 10.9 on a mac. The error on screen was “This copy of the Install OS X Mavericks application can’t be verified..”

The fix is relatively easy in this case. In the main menu when you first boot onto the USB drive (the one that has the option for disk utility, install etc), from the top of the screen menu, select utilities and then terminal.

Set the current date as such (in this example lets say the date is 3/14/2014 at 9:00am

date 031409002014

Now exit the terminal by typing “exit” and then try to re-install as normal and you should be good to go.

Openindiana Public key SSH issue

Wed, 08 Jan 2014 13:46:56 -0800

I thought I would throw this one out there as it took me about 30 min to figure this out.
I had an OpenIndiana server that I was working on and I wanted to make a new user that had a public key for access. The problem was that when I tried to connect, it would just come back with the usual "Permission denied (publickey)." error.
Normally, this is because you forgot to set the key permissions correctly:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

But in this case that was not the problem. I even tried to copy the keys to a windows host, used puttygen to convert the key and then putty to load it which gave the following error:

"Server refused public-key signature despite accepting key!"

WTF?
By default, OI does not log like linux does so I had to enable auth logging:

vi /etc/syslog.conf

(uncomment the following line)
auth.notice                     ifdef(`LOGHOST', /var/log/authlog, @loghost)

Now just restart the syslog service:

svcadm restart system-log

Now if you tail -f the authlog file you should see some details:

root@zfs-server:~# tail -f /var/log/authlog
Jan  8 18:29:48 zfs-server sshd[6387]: [ID 7311328 auth.notice] pam_unix_account: sshd-pubkey attempting to validate locked account johnny from 10.1.1.50
Jan  8 18:29:48 zfs-server sshd[6387]: [ID 8000347 auth.notice] Failed publickey for johnny from 10.1.1.50 port 64885 ssh2

See that, it looks like the account was just locked out! Pretty easy to fix this:

passwd -u johnny

That's all it took, hope this helps another admin, if so please feel to let others know in the comments below.

How to kill your camera in Mavericks

Thu, 02 Jan 2014 11:26:18 -0800

There have been a number of "fun" viruses and other code as of late that try to use your camera in your apple computer. A while back a group of folks at techslaves.org created a tool that would disable the ichat camera (that little camera in your macbook or iMac). The problem is that they have not updated their code in a while and there was not really a good solution for Mavericks.

Today that changes, I'm releasing the following simple script that should help Mavericks users disable their cameras. You can download the file here.

To use the script, simply download the file, then open Applications > Utilities > Terminal.

In terminal just type:

cd ~/Downloads
unzip kill-cam.zip
chmod 755 kill-cam.sh
./kill-cam.sh

This will prompt you to press either "1" or "2" depending on if you want to enable or disable the camera.

If you want to change (read re-enable) your camera, all you need to do is launch terminal again and type:

cd ~/Downloads
./kill-cam.sh

Then select "2".

Leave a comment below and let me know if this works for you in 10.9.

For the un-trusting type (like me) here is the script:

#!/bin/bash

declare -a arr=(/Library/CoreMediaIO/Plug-Ins/DAL/AppleCamera.plugin/Contents/MacOS/AppleCamera /Library/CoreMediaIO/Plug-Ins/FCP-DAL/AppleCamera.plugin/Contents/MacOS/AppleCamera /System/Library/Frameworks/CoreMediaIO.framework/Versions/A/Resources/VDC.plugin/Contents/MacOS/VDC)

trigger () {
for i in ${arr[@]}; do 
 sudo chmod $1 $i
done
}
 
printf "(1) Kill camera \n(2)enable camera \n"
read coms
if [ $coms -eq 1 ]; then
 trigger 000;
else 
 trigger 755; 
fi

Install Puppet on CentOS 6.4

Fri, 26 Apr 2013 10:54:05 -0700

Installing Puppet on CentOS 6.4

Before I start I want to give credit to this site for starting me off in the right direction. His/Her instructions ALMOST work, but fail to finish the job for Centos 6.4 and Puppet 3.x. 6tech.org

Before we start, there are assumptions being made. First, you are starting from a vanilla minimal install of CentOS 6.4. Secound, when I say "FQDN" that means replace that with the fully qualified domain name of the host. IE Server.internal.net

First you need to grab the appropriate puppet installer for your OS (x86_64 or i386)

http://yum.puppetlabs.com/el/6/products/

wget http://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

wget http://yum.puppetlabs.com/el/6/products/i386/puppetlabs-release-6-7.noarch.rpm

rpm -ivh puppetlabs-release-6-7.noarch.rpm
yum clean all
yum update

Now lets get everything for the install:

yum groupinstall "Development tools"
yum install -y httpd httpd-devel mod_ssl ruby-devel rubygems gcc-c++ curl-devel zlib-devel openssl-devel puppet-server

Now start the Puppet-Server

/etc/init.d/puppetmaster start

Set Puppet Master to run on start-up

puppet resource service puppetmaster ensure=running enable=true

Configure Puppet and Apache server:

vim /etc/httpd/conf.d/puppetmaster.conf
Replace "your-fqdn". With your servers fully qualified domain name
	# RHEL/CentOS:
LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.19
PassengerRuby /usr/bin/ruby
 
# And the passenger performance tuning settings:
PassengerHighPerformance On
PassengerUseGlobalQueue On
# Set this to about 1.5 times the number of CPU cores in your master:
PassengerMaxPoolSize 6
# Recycle master processes after they service 1000 requests
PassengerMaxRequests 1000
# Stop processes if they sit idle for 10 minutes
PassengerPoolIdleTime 600
 
Listen 8140
<VirtualHost *:8140>
    SSLEngine On
 
    # Only allow high security cryptography. Alter if needed for compatibility.
    SSLProtocol             All -SSLv2
    SSLCipherSuite          HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP
    SSLCertificateFile      /var/lib/puppet/ssl/certs/<strong><your-fqdn\></strong>.pem
    SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/<strong><your-fqdn></strong>.pem
    SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
    SSLVerifyClient         optional
    SSLVerifyDepth          1
    SSLOptions              +StdEnvVars +ExportCertData
 
    # These request headers are used to pass the client certificate
    # authentication information on to the puppet master process
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
 
    RackAutoDetect On
    DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
    <Directory /usr/share/puppet/rack/puppetmasterd/>
        Options None
        AllowOverride None
        Order Allow,Deny
        Allow from All
    </Directory>
</VirtualHost>

Start up Apache:

etc/init.d/httpd restart

Disable WEBrick and enable Apache on boot:

chkconfig puppetmaster off
chkconfig httpd on

Make sure the port is open and it’s listening:

netstat -ln | grep 8140
1	tcp    0  0 0.0.0.0:8140         0.0.0.0:*              LISTEN

Set the server to auto-sign certs. (If you are worried about security, don't do this, use puppet cert --sign FQDN).

*Append this to the end of the file

vim /etc/puppet/puppet.conf
[master]
certname = puppet-server #Use the FQDN here
autosign = true

Client Node install

Add the puppet labs repo

rpm -ivh http://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm

Install the Puppet Client

yum install -y puppet

If you are not using DNS in your envrionment, you will need to manually edit your hosts file.

vim /etc/hosts
192.168.x.x                node
192.168.x.y                puppet-server

Edit /etc/puppet/puppet.conf and add the agent variables:

vim /etc/puppet/puppet.conf
# In the [agent] section
 
    server = puppet-server #Should be the FQDN!
    report = true
    pluginsync = true

Set the puppet agent to run on boot:

chkconfig puppet on
puppet agent --daemonize

Now test the client:

puppet agent --t

That should connect you to the server which will automatically sign the cert. If you have opted to manually sign, you now need to go back to the server and run.

puppet cert --sign FQDN

Did this work for you? Let others know in the comments below!

Installing NGINX on CentOS 6.4

Thu, 04 Apr 2013 12:04:39 -0700

Installing NGINX is pretty easy on Cent, we just need to make the following changes.

Create a new file called /etc/yum.repos.d/nginx.repo and append the following to it

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

Now run yum update and then yum install nginx

Tada!

If this was helpful, let others know in the comments below.

Install Nagios 3.5 on CentOS 6.4

Thu, 04 Apr 2013 10:51:50 -0700

Below I am going to outline how to install Nagios 3.5.0 on CentOS 6.4 from source. I'm assuming that you are starting from a clean minimal install of CentOS

Ok first lets get the needed packages

yum install wget httpd php gcc glibc glibc-common gd gd-devel make net-snmp libssl-dev openssl* wget policycoreutils-python

Now lets get nagios and it's plugins, at this time the link below was the most up to date, but you can verify this by omitting the data after the last trailing slash.

cd /tmp
wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.5.0.tar.gz
wget http://sourceforge.net/projects/nagiosplug/files/nagiosplug/1.4.16/nagios-plugins-1.4.16.tar.gz

Now unpack and lets start with Nagios fist:

gunzip *.gz
tar xf nagios-3.5.0.tar 
cd nagios
/usr/sbin/adduser nagios
/usr/sbin/groupadd nagcmd
usermod -a -G nagcmd nagios
./configure --with-command-group=nagcmd 

make all
make install 
make install-init 
make install-config 
make install-commandmode 
make install-webconf 

chkconfig --add nagios
chkconfig --level 35 nagios on
chkconfig --add httpd
chkconfig --level 35 httpd on
#Create a nagios login
htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Now lets get those plugins working.

cd /tmp
tar xf nagios-plugins-1.4.16.tar
cd nagios-plugins-1.4.16
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install

Now check your base build, if you get any errors, you will need to fix them first

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Now start it up

service nagios start
service httpd start

#Add port 80 to your iptables
vi /etc/sysconfig/iptables

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT

#Now restart
service iptables restart

Now, here is where I want to strangle most other websites... DO NOT disable selinux! It's dumb to not just properly configure it! Below I will help you to authorize Nagios in SElinux.

chcon -R -t httpd_sys_content_t /usr/local/nagios
set enforce 0
chcon -R -t httpd_sys_script_t /usr/local/nagios/var/rw/nagios.cmd 
set enforce 1

If you find that you are getting blocked cmd in nagios, run the following to fix it.

grep denied /var/log/audit/audit.log |audit2why
audit2allow -a
#For each context header (IE httpd_sys_script_t) run
audit2allow -a -M httpd_sys_script_t
#Then activate each header
semodule -i httpd_sys_script_t.pp
#repeat

That should do it, to add hosts, you need to edit the files in /usr/local/nagios/etc/objects/

If this was helpful, please let others know in the comments below!