Whirr Trouble Shooting

This post contains some troubleshooting tips which hopefully should help you get out of troubled waters when you install/use Whirr on CloudStack.

• Case 1

  java.util.concurrent.ExecutionException: java.io.IOException: Too many instance failed while bootstrapping! 0 successfully started instances while 2 instances failed.

  If you instances go through these states(starting, running, destroying, expunging) quickly, it is highly probably that whirr cannot login to the instances to bootstrap them. If the template you use has a hardcoded username/password, add the following line to your cluster definition file to make Whirr aware of that.

  whirr.bootstrap-user=your_user_name:your_password
  

  The following exceptions also imply this type of issue.

  org.jclouds.rest.AuthorizationException: (root:pw[63d90337b21005ea9a4bb6d617d4e54e]@10.244.18.65:22) (root:pw[63d90337b21005ea9a4bb6d617d4e54e]@10.244.18.65:22) error acquiring {hostAndPort=10.244.18.65:22, loginUser=root, ssh=null, connectTimeout=60000, sessionTimeout=60000} (out of retries - max 7): Exhausted available authentication methods

  java.lang.NullPointerException: no credential found for root on node 206224c5-756a-43a4-9925-b4916a1cd585

• Case 2

  java.util.concurrent.ExecutionException: org.jclouds.cloudstack.AsyncJobException: job AsyncJob{accountId=789c4968-e835-11e2-890d-0023ae94f722, cmd=org.apache.cloudstack.api.command.user.vm.DeployVMCmd, created=Wed Jul 10 11:03:50 EDT 2013, id=f6c6ebde-2694-4c55-82c2-8b6305861d1b, instanceId=null, instanceType=null, progress=0, result=null, resultCode=FAIL, resultType=object, status=FAILED, userId=78a3f050-e835-11e2-890d-0023ae94f722, error=AsyncJobError{errorCode=INSUFFICIENT_CAPACITY_ERROR, errorText=Unable to create a deployment for VM[User|you-makes-me-stronger-e6d]}} failed with exception AsyncJobError{errorCode=INSUFFICIENT_CAPACITY_ERROR, errorText=Unable to create a deployment for VM[User|you-makes-me-stronger-e6d]}

  This implies that the host does not have enough capacity to launch VM. Go to CloudStack Management server UI → Dashboard → system capacity → fetch latest to check the low available resources. Maybe you are running out of IPs that can be assigned or have less secondary storage available. Also, this type of error happens frequently for users who deploy CloudStack on a small scale and have many failed launching attempts already. By default, if you failed to launch a VM, CloudStack will wait for 24 hours before reclaiming the resources assigned to the failed VMs. Go to CloudStack Management server UI → Global settings and change the value of expunge.delay and expunge.interval to a smaller value to ease the case.
• Case 3

  com.cloud.exception.ResourceUnavailableException: Resource [Pod:1] is unreachable: Unable to apply dhcp entry on router 
  

  These error(s) occurred because there was an unknown DHCP Server on the same subnet as the system vm, but that unknown DHCP Server was not included in the CSMS IP Address Space for management devices.If there is no other DHCP server, simply restart the Virtual Router.
• Last but not least, the cloudstack management-server.log and whirr.log are the main logs to help you track errors.