Community
 
 
 

CloudPlatform 3.x

343 abonnés
 
Avatar
Pankaj Paliwal

System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation

Avatar

System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation

I am attempting to setup a trial installation following the steps followed in the CloudPlatform Master Class (and Trial Installation Guide). After pressing the Launch button, the operation never completes (i.e. the progress indication continues to spin and no success message is displayed). When I subsequently log into CloudPlatform UI and inspect the Infrastructure, I see that the status for the System VMs is Starting.

i've attached the management-server.log and catalina.out file where i see things like:

INFO [xen.resource.CitrixResourceBase] (DirectAgent-10:) Programmed default network rules for s-1-VM
WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) Unable to ssh to the VM: Can not ping System vm s-1-VMdue to:Timeout, Unable to logon to 169.254.1.143
INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) The guru did not like the answers so stopping VM[SecondaryStorageVm|s-1-VM]
INFO [xen.resource.CitrixResourceBase] (DirectAgent-39:) Removed network rules for vm s-1-VM
ERROR [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Failed to start instance VM[SecondaryStorageVm|s-1-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[SecondaryStorageVm|s-1-VM] due to error in finalizeStart, not retrying

(stack trace follows)

INFO [xen.resource.CitrixResourceBase] (DirectAgent-10:) Programmed default network rules for s-1-VM
WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) Unable to ssh to the VM: Can not ping System vm s-1-VMdue to:Timeout, Unable to logon to 169.254.1.143
INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) The guru did not like the answers so stopping VM[SecondaryStorageVm|s-1-VM]
INFO [xen.resource.CitrixResourceBase] (DirectAgent-39:) Removed network rules for vm s-1-VM
ERROR [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Failed to start instance VM[SecondaryStorageVm|s-1-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[SecondaryStorageVm|s-1-VM] due to error in finalizeStart, not retrying

(more stack trace)

The console window for the VMs indicate that the VMs come up but a number of errors are displayed including the following:

Starting enhanced syslogd: rsyslogd.
Not starting as we're not running in a vm.
Starting ACPI services...RTNETLINK1 answers: No such file or directory
acpid: error talking to the kernel via netlink
.
Starting the system activity data collector: sadc.
Starting DNS forwarder and DHCP server: dnsmasq
dnsmasq: unknown interface eth0
failed!
Starting OpenBSD Secure Shell server: sshd.Starting enhanced syslogd: rsyslogd.
Not starting as we're not running in a vm.
Starting ACPI services...RTNETLINK1 answers: No such file or directory
acpid: error talking to the kernel via netlink
.
Starting the system activity data collector: sadc.
Starting DNS forwarder and DHCP server: dnsmasq
dnsmasq: unknown interface eth0
failed!
Starting OpenBSD Secure Shell server: sshd.
Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
(99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80
no listening sockets available, shutting down
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.
failed!
Starting periodic command scheduler: cron.
Starting OpenBSD Secure Shell server: sshd.
Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
(99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80
no listening sockets available, shutting down
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.
failed!
Starting periodic command scheduler: cron.
Starting OpenBSD Secure Shell server: sshd.
Starting web server: apache2apache2: apr_sockaddr_info_get() failed for systemvm
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
[Tue Jun 25 12:31:17 2013] [error] (EAI 3)Temporary failure in name resolution: Failed to resolve server name for 10.1.1.1 (check DNS) -- or specify an explicit ServerName
(99)Cannot assign requested address: make_sock: could not bind to address 10.1.1.1:80
no listening sockets available, shutting down
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.

Attached Files


Jim Glennon MEMBERS
10 commentaires
0

Vous devez vous connecter pour laisser un commentaire.

 
 

Previous 10 commentaires

Avatar
Pankaj Paliwal
Avatar

System VMs not successfully starting with CloudPlatform 3.0.6/XenServer 6.1) Trial Installation

I've been looking at the logs a bit more and it seems like this is some kind of network setup issue since the management server is not able to ping the System VMs (i tried from the console window as well and was not able to). I'm wondering if there is more info in the logs on the System VMs that might be helpful. Does anyone know what the root password is to access them?


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

The root password isn't available, but you can login to your system VMs using an SSH key.

See http://www.tutkiun.com/2012/06/login-to-system-vmsrouters-cloudstack.html for more details.

--Mike


Mike Little MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

Thanks for this info. Unfortunately I get a "No route to host" error when i try this. Perhaps the problems I am encountering are with my XenServer host configuration?


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

Actually this seems more like a problem with the link local network (if that's what it's called) that CloudPlatform is setting up. Not sure how to troubleshoot this....


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

my suspicions were correct about the local link network. i started from scratch and see the following error when trying to start the system VMs. still don't know wtf to do about this though....

2013-06-28 08:01:30,731 WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:null) Unable to create local link networkThe server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem. at com.xensource.xenapi.Types.checkResponse(Types.java:1514) at com.xensource.xenapi.Connection.dispatch(Connection.java:372) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:906) at com.xensource.xenapi.VIF.plug(VIF.java:776) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4399) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2911) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:423) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:55)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:192) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

it appears that this has been reported over at Apache. perhaps i should back off from XenServer 6.1?

http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-issues/201303.mbox/%3CJIRA.12631888.1360646963600.383320.1362494721443@arcas%3E


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

Hi, I haven't seen this issue before. Do you have any hotfixes installed on your XenServer 6.1 host? The latest ones aren't always supported. For XS 6.1 I think hotfixes up to XS61E019 are currently supported. Another thing to try is to install a CloudPlatform patch release.

CloudPlatform 3.0.6 PatchD (RHEL 6.3)
CloudPlatform 3.0.7 PatchA (RHEL 6.3)

These have many bugfixes so there is a chance the issue you are seeing is already resolved.

Best regards,

{color:#555555}Kirk Kosinski{color}
{color:#999999}MCITP: EA / VA / EDA7, VCP 4 / 5, CCA{color}


Kirk Kosinski CITRIX EMPLOYEES
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

Also, if this is a basic zone, make sure to disable Open vSwitch (xe-switch-network-backend bridge) and reboot.

Best regards,

{color:#555555}Kirk Kosinski{color}
{color:#999999}MCITP: EA / VA / EDA7, VCP 4 / 5, CCA{color}


Kirk Kosinski CITRIX EMPLOYEES
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

this seemed to do the trick. i installed the recommended patches for both and was successful in getting past this problem. thanks!


Jim Glennon MEMBERS
Actions pour les commentaires Permalien
Avatar
Pankaj Paliwal
Avatar

Hi All:
I have solved the issue ....First,, I used XenCenter add the Host to pool and ignored the primary storage can't
mount error.I used CCP UI add the host to cluster again,the error would be removed in a few minutes...


robin dun MEMBERS
Actions pour les commentaires Permalien

Top Contributors