System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation
System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation
I am attempting to setup a trial installation following the steps followed in the CloudPlatform Master Class (and Trial Installation Guide). After pressing the Launch button, the operation never completes (i.e. the progress indication continues to spin and no success message is displayed). When I subsequently log into CloudPlatform UI and inspect the Infrastructure, I see that the status for the System VMs is Starting.
i've attached the management-server.log and catalina.out file where i see things like:
INFO [xen.resource.CitrixResourceBase] (DirectAgent-10:) Programmed default network rules for s-1-VM WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) Unable to ssh to the VM: Can not ping System vm s-1-VMdue to:Timeout, Unable to logon to 169.254.1.143 INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) The guru did not like the answers so stopping VM[SecondaryStorageVm|s-1-VM] INFO [xen.resource.CitrixResourceBase] (DirectAgent-39:) Removed network rules for vm s-1-VM ERROR [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Failed to start instance VM[SecondaryStorageVm|s-1-VM] com.cloud.utils.exception.ExecutionException: Unable to start VM[SecondaryStorageVm|s-1-VM] due to error in finalizeStart, not retrying
(stack trace follows)
INFO [xen.resource.CitrixResourceBase] (DirectAgent-10:) Programmed default network rules for s-1-VM WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) Unable to ssh to the VM: Can not ping System vm s-1-VMdue to:Timeout, Unable to logon to 169.254.1.143 INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) The guru did not like the answers so stopping VM[SecondaryStorageVm|s-1-VM] INFO [xen.resource.CitrixResourceBase] (DirectAgent-39:) Removed network rules for vm s-1-VM ERROR [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Failed to start instance VM[SecondaryStorageVm|s-1-VM] com.cloud.utils.exception.ExecutionException: Unable to start VM[SecondaryStorageVm|s-1-VM] due to error in finalizeStart, not retrying
(more stack trace)
The console window for the VMs indicate that the VMs come up but a number of errors are displayed including the following:
Starting enhanced syslogd: rsyslogd. Not starting as we're not running in a vm. Starting ACPI services...RTNETLINK1 answers: No such file or directory acpid: error talking to the kernel via netlink . Starting the system activity data collector: sadc. Starting DNS forwarder and DHCP server: dnsmasq dnsmasq: unknown interface eth0 failed! Starting OpenBSD Secure Shell server: sshd.Starting enhanced syslogd: rsyslogd. Not starting as we're not running in a vm. Starting ACPI services...RTNETLINK1 answers: No such file or directory acpid: error talking to the kernel via netlink . Starting the system activity data collector: sadc. Starting DNS forwarder and DHCP server: dnsmasq dnsmasq: unknown interface eth0 failed! Starting OpenBSD Secure Shell server: sshd. Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName (99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80 no listening sockets available, shutting down Unable to open logs Action 'start' failed. The Apache error log may have more information. failed! Starting periodic command scheduler: cron. Starting OpenBSD Secure Shell server: sshd. Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName (99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80 no listening sockets available, shutting down Unable to open logs Action 'start' failed. The Apache error log may have more information.: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName Unable to open logs Action 'start' failed. The Apache error log may have more information. failed! Starting periodic command scheduler: cron. Starting OpenBSD Secure Shell server: sshd. Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName [Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName (99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80 no listening sockets available, shutting down Unable to open logs Action 'start' failed. The Apache error log may have more information.
System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation
I've been looking at the logs a bit more and it seems like this is some kind of network setup issue since the management server is not able to ping the System VMs (i tried from the console window as well and was not able to). I'm wondering if there is more info in the logs on the System VMs that might be helpful. Does anyone know what the root password is to access them?
Thanks for this info. Unfortunately I get a "No route to host" error when i try this. Perhaps the problems I am encountering are with my XenServer host configuration?
Actually this seems more like a problem with the link local network (if that's what it's called) that CloudPlatform is setting up. Not sure how to troubleshoot this....
my suspicions were correct about the local link network. i started from scratch and see the following error when trying to start the system VMs. still don't know wtf to do about this though....
2013-06-28 08:01:30,731 WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:null) Unable to create local link networkThe server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem. at com.xensource.xenapi.Types.checkResponse(Types.java:1514) at com.xensource.xenapi.Connection.dispatch(Connection.java:372) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:906) at com.xensource.xenapi.VIF.plug(VIF.java:776) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4399) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2911) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:423) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:55) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:192) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679)
Hi, I haven't seen this issue before. Do you have any hotfixes installed on your XenServer 6.1 host? The latest ones aren't always supported. For XS 6.1 I think hotfixes up to XS61E019 are currently supported. Another thing to try is to install a CloudPlatform patch release.
Hi All: I have solved the issue ....First,, I used XenCenter add the Host to pool and ignored the primary storage can't mount error.I used CCP UI add the host to cluster again,the error would be removed in a few minutes...
