VMware ESX Corrupt --redo log--

I was recently faced with a VMware ESX issue and while researching a solution I found an useful article in the VMware knowledge base.

Issue:

I notice one of the Servers weren't functioning correctly and after further investigation of the logs I notice this following error message "The Redo Log on Srvr01-0000001.vmdk has detected to be corrupt. The virtual machine needs to be powered down. If the problem still persists, you need to discard the redo log". After attempting a few times to restart the guest OS and failing, I use VMware article which resolved this issues for me.

The article is well written so the only thing I would like to add is the followings steps for creating a local user account on the VMware host. This account is needed to ssh into the host and perform the steps necessary to resolve this issue. In addition Root login via SSH is disabled by default so these steps are necessary.

Steps to create a local user with shell access on a VMware ESX host:

Step One:

Launch your VMware infrastructure client and connect to the ESX server that’s hosting the problematic guest OS, if you connect to your virtual center server you wouldn’t get the option to create a local user. Login with your Root user and and password.

Step two:

Once you are logged in click on the "Users and Groups" tab, insure that you are in the "users view", then right click a user account from the list select add and fill in requested information (username and password).

***note** Check off the box that says "Grant shell access to this user"


You can now ssh to the ESX server using your newly created credentials. Once you log in issue the "su -" command, you will be prompted for your root password enter it and you are ready to continue with the rest of the article.

Excerpt from the article:

To terminate the Master World and User Worlds for the virtual machine:

1. Run the following command to list the running virtual machines to determine the virtual machine ID for the affected virtual machine:

#cat /proc/vmware/vm/*/names

The output appears similar to:

vmid=1076 pid=-1 cfgFile="/vmfs/volumes/50823edc-d9110dd9-8994-9ee0ad055a68/VMNAME/VMNAME.vmx" uuid="50 28 4e 99 3d 2b 8d a0-a4 c0 87 c9 8a 60 d2 31" displayName="VMNAME-192.168.1.10"

Note: vmid='1076' is used as an example in this article.

2. Run the following command to identify the Master World ID:

# less -S /proc/vmware/vm/1076/cpu/status

Expand the terminal or scroll until you can see the right-most column labeled 'group'. In this column you find the vm.####

In this example, '1092' is the ID of the Master World.

3. Run the following command to terminate the Master World and the virtual machine running in it:

/usr/lib/vmware/bin/vmkload_app -k9 1092

4. The virtual machine's User Worlds and the virtual machine's processes are stopped.
If the command is successful, you see output similar to:

# /usr/lib/vmware/bin/vmkload_app --kill 9 1070 Warning: Jul 12 07:24:06.303: Sending signal '9' to world 1070.

If the Master World ID is wrong, you see the error:

# /usr/lib/vmware/bin/vmkload_app --kill 9 1071

Warning: Jul 12 07:21:05.407: Sending signal '9' to world 1071.
Warning: Jul 12 07:21:05.407: Failed to forward signal 9 to cartel 1071: 0xbad0061

The virtual machine is now powered off. Power on the virtual machine. Verify that it is able to boot properly and that the message error no longer occurs.

Here is the link to the article --> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006585.

Winter Cleaning "Rebuilding my HackLab"

Its that time of year again as a Tech when you have to take a look around and figure out which junk you are going to keep and which ones you are going to throw away. I have been meaning to rebuild my home network and make it half "HackLab" and half productions.

In doing so I started to ask myself if I wanted to keep VMWare or move on to Citrix XenServer, I will do a full blog post later with the details of that decision. I will also include some information on my setup once its complete, for now here is some pics of my old "hacklab"

My Command Center

1 HP DL 380 G4, 1 SupperMicro,2 HP DL 360 G3 servers

7 Cisco 2600 Routers, 1 Catalyst 2900 and 1 Catalyst 3500 switch

Laptops and PC spare parts

Once I have my lab in order I need to figure out which items to put on Ebay :)

Free Malware Analysis Services

While analyzing an infected machine today I retrieved a sample that needed further analysis. I would normally start by scanning these files with my Anti-virus software,however since the Anti-virus was disabled by the virus that wouldn’t have been too helpful. I currently don’t have a test lab to analyze these samples so the following services below came in handy.

Some of them are only capable of analyzing potentially infected files, while others can also analyze a URL. The reports are pretty details and saved me a lot of time.

http://scanner.novirusthanks.org/ (file scanner and URL scanner)

http://www.virustotal.com (file scanner)

http://anubis.iseclab.org/ (file scanner and URL scanner)

http://virusscan.jotti.org/ (file scanner)

http://filterbit.com/ (file scanner)

http://camas.comodo.com/ (file scanner)

http://www.cwsandbox.org/?page=submit (file scanner)

http://eureka.cyber-ta.org/ (file scanner)

http://www.joebox.org/submit.php (file scanner and URL scanner)

http://www.norman.com/security_center/security_tools/submit_file/en (file scanner)

http://www.threatexpert.com/submit.aspx (file scanner)

http://xandora.security.net.my/?page_id=332 (file scanner)

Omeka Install issue & resolution

I was tasked with setting up a test Omeka Server for our Library Archivist, based on the installation section on the site this should take about “five-minutes” give or take.

What is Omeka

Omeka is a free and open source collections based web-based publishing platform for scholars, librarians, archivists, museum professionals, educators, and cultural enthusiasts. Its “five-minute setup” makes launching an online exhibition as easy as launching a blog. Omeka is designed with non-IT specialists in mind, allowing users to focus on content and interpretation rather than programming. It brings Web 2.0 technologies and approaches to academic and cultural websites to foster user interaction and participation. It makes top-shelf design easy with a simple and flexible templating system. Its robust open-source developer and user communities underwrite Omeka’s stability and sustainability.

What are the minimum and the recommended specifications?

• Linux operating system
• Apache HTTP server (with mod_rewrite enabled)
• MySQL version 5.0 or greater
• PHP scripting language version 5.2.4 or greater (with mysqli and exif extensions installed)
• ImageMagick image manipulation software (for resizing images in Omeka)

Direct Link to Omeka Documentation
http://omeka.org/codex/Documentation

Omeka Plugins
http://omeka.org/add-ons/plugins

Installation
http://omeka.org/codex/Installation

http://digin.arizona.edu/files/omeka.pdf


The above instruction goes under the assumption that you already have a LAMP environment in place. Since I haven’t done a LAMP setup in quite some time I did a google.com search and landed on one of my favorite tutorial site http://www.howtoforge.com/ubuntu_lamp_for_newbies.

One small issue I encountered and how I resolved it

After doing the LAMP setup and following both guides under the installation section, I connected to the web server URL http://server and instead of displaying, I was prompted to download a file “index.php”.

Back to google.com…After searching going through the various Apache configuration folders and directories, I realize that I needed to add the below configuration to the bottom of the mime.conf file just above the last "<\IfModule>" closing tag.

Steps taken:

Logged into the Linux Web Server and made the following changes;

ituser@TestSrvr:/$ cd /etc/apache2/mods-enabled
ituser@TestSrvr:/$ sudo nano ./mime.conf

Open IfModule mod_mime.c

AddType application/x-httpd-php .php

AddType application/x-httpd-php .phtml
AddType application/x-httpd-php .php3
AddType application/x-httpd-php .php4
AddType application/x-httpd-php .html
AddType application/x-httpd-php-source .phps
Close IfModule

Reference --> http://forums.digitalpoint.com/showthread.php?t=7584

Restarted apache “sudo /etc/init.d/apache2 restart” and I was ready to rock-n-roll!

Self Introduction

Being that this is my first post I figure I should start off by introducing myself. I am a Jr. Network Admin at a College, Husband, father, and I also run my own IT consulting business on the side. I have a strong interest in Information Security but I am currently not in that field as yet.

The reason I decided to start this blog is to basically keep track of projects I am working on, things of interest to me, and of course all of those wonderful links and how-to's that you come across when troubleshooting an issue but two months later can’t find.

The type of posting you can look forward too could include topics such as, “the steps I took to deploy a Linux web Server at work", "various topics on VMware deployment and upgrades", "setting up a small office network and securing it”. Or just things I came across while on site at a client.

Hope you keep reading and of course comments are always welcome!