Community
 
 
 

CloudPlatform 3.x

343 followers
 
Avatar
Administrator

com.cloud.utils.exception.CloudRuntimeException: can't mount Secondary Storage from HyperVisor

Avatar

com.cloud.utils.exception.CloudRuntimeException: can't mount Secondary Storage from HyperVisor

Im running into an annoying issue when creating a new template(by using Cloudstack, template, register new template) and was wondering if anyone has seen this.
Below are the error logs and mysql changes made to make this work. I just don't understand why this happens when some templates are uploaded. The version I'm running is CloudPLatform 3.0.3 and Xenserver 6.0.3 with updates XS602E001-010.
The issue is that when a new template is uploaded and a new instance is being created I get an error that the mount point can't be mounted on the hypervisor, the SSVM has download the template successfully and can mount the SR. Its when the Hypervisor tries to mount the SR(install_path) where it errors out. As you can see the error in bold show that there is a mount occurring with "//" in the mount path. If anyone has any ideas on why this is happening it would be great to know. Thanks in advanced.

2012-12-21 11:44:34,346 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-369:null) destoryVDIbyNameLabel failed due to there are 0
VDIs with name cloud-4c12fabd-3918-48c6-b079-a2e1728c1b3c
2012-12-21 11:44:34,346 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-369:null) can't mount
172.20.4.44:/NFS_DISK_1/secondary/template/tmpl//2/235/ to
/var/run/cloud_mount/e96fbe90-f0bf-41f7-b15f-46ecb9af5cce
2012-12-21 11:44:34,346 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-369:null) Catch Exception
com.cloud.utils.exception.CloudRuntimeException on
host:05ff59f1-f964-4aab-97a8-a198b77744b8 for template:
*nfs://172.20.4.44/NFS_DISK_1/secondary/template/tmpl//2/235//a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd <http://172.20.4.44/NFS_DISK_1/secondary/template/tmpl//2/235//a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd>*
due to com.cloud.utils.exception.CloudRuntimeException: can't mount
172.20.4.44:/NFS_DISK_1/secondary/template/tmpl//2/235/ to
/var/run/cloud_mount/e96fbe90-f0bf-41f7-b15f-46ecb9af5cce
com.cloud.utils.exception.CloudRuntimeException: can't mount
172.20.4.44:/NFS_DISK_1/secondary/template/tmpl//2/235/ to
/var/run/cloud_mount/e96fbe90-f0bf-41f7-b15f-46ecb9af5cce
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.copy_vhd_from_secondarystorage(CitrixResourceBase.java:2458)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2480)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:456)
at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-12-21 11:44:34,347 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-369:null) Seq 2-1548493196: Response Received:
2012-12-21 11:44:34,347 DEBUG [agent.transport.Request]
(DirectAgent-369:null) Seq 2-1548493196: Processing: { Ans: , MgmtId:
59490837860857, via: 2, Ver: v1, Flags: 110,
[{"storage.PrimaryStorageDownloadAnswer":{"templateSize":0,"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException on
host:05ff59f1-f964-4aab-97a8-a198b77744b8 for template:
nfs://172.20.4.44/NFS_DISK_1/secondary/template/tmpl//2/235//a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd <http://172.20.4.44/NFS_DISK_1/secondary/template/tmpl//2/235//a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd>
due to com.cloud.utils.exception.CloudRuntimeException: can't mount
172.20.4.44:/NFS_DISK_1/secondary/template/tmpl//2/235/ to
/var/run/cloud_mount/e96fbe90-f0bf-41f7-b15f-46ecb9af5cce","wait":0}}]
}

When looking at the database I see that there are double forward slashes("//") for the mount path and visible in the error above also. The database looks like this:

mysql> select * from template_host_ref where id = 230\G
************************** 1. row **************************
id: 230
host_id: 16
template_id: 235
created: 2012-12-21 18:21:14
last_updated: 2012-12-21 18:31:39
job_id: 00f37d10-f856-4975-9073-95da888c82f9
download_pct: 100
size: 53687091200
physical_size: 1515106816
download_state: DOWNLOADED
error_str: Install completed successfully at 12/21/12 6:31 PM
local_path:
/mnt/SecStorage/61c5beae-a666-3e8c-a4a3-0975ee1dc6a9/template/tmpl/2/235/dnld6807828163053471184tmp_
install_path: template/tmpl//2/235//a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd
url:
http://172.20.1.151/templates/209e79e4-3bec-874f-541b-5ab11ea07eaa/centos63.vhd <http://172.20.1.151/templates/209e79e4-3bec-874f-541b-5ab11ea07eaa/centos63.vhd>
destroyed: 0
is_copy: 0
1 row in set (0.00 sec)

Now I update the install_path where I remove the double slashes:

mysql> update template_host_ref set install_path ="template/tmpl/2/235/a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd" where id = 230;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from template_host_ref where id = 230\G
************************** 1. row **************************
id: 230
host_id: 16
template_id: 235
created: 2012-12-21 18:21:14
last_updated: 2012-12-21 18:31:39
job_id: 00f37d10-f856-4975-9073-95da888c82f9
download_pct: 100
size: 53687091200
physical_size: 1515106816
download_state: DOWNLOADED
error_str: Install completed successfully at 12/21/12 6:31 PM
local_path:
/mnt/SecStorage/61c5beae-a666-3e8c-a4a3-0975ee1dc6a9/template/tmpl/2/235/dnld6807828163053471184tmp_
install_path: template/tmpl/2/235/a7574a72-d2b3-35ae-9671-ee2fe6eea865.vhd
url:
http://172.20.1.151/templates/209e79e4-3bec-874f-541b-5ab11ea07eaa/centos63.vhd <http://172.20.1.151/templates/209e79e4-3bec-874f-541b-5ab11ea07eaa/centos63.vhd>
destroyed: 0
is_copy: 0
1 row in set (0.00 sec)

After the update the VM create instance works with out issues:

2012-12-21 12:41:16,922 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) STATS: Can alloc
CPU from host: 1, used: 23000, reserved: 0, actual total: 85240, total
with overprovisioning: 85240; requested
cpu:9200,alloc_from_last_host?:false ,considerReservedCapacity?: true
2012-12-21 12:41:16,922 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) STATS: Can alloc
MEM from host: 1, used: 21474836480, reserved: 0, total: 134368347456;
requested mem: 8589934592 <tel:8589934592> ,alloc_from_last_host?:false
,considerReservedCapacity?: true
2012-12-21 12:41:16,922 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) Found a suitable
host, adding to list: 1
2012-12-21 12:41:16,926 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) Checking if host: 2
has enough capacity for requested CPU: 9200 and requested RAM:
8589934592 <tel:8589934592> , cpuOverprovisioningFactor: 1.0
2012-12-21 12:41:16,932 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) Hosts's actual
total CPU: 85240 and CPU after applying overprovisioning: 85240
2012-12-21 12:41:16,932 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-95:job-622 FirstFitRoutingAllocator) Free CPU: 83740 ,
Requested CPU: 9200

Attached Files


Edward Valencia MEMBERS
3 comments
0

Please sign in to leave a comment.

 
 

Previous 3 comments

Avatar
Martin Gucky

Hi,
are you able to mount your secondary storage manually directly from xenserver? Log in to xenserver host via SSH and try using the mount command. I think the command shoudl be "mount -t nfs 172.20.4.44:/NFS_DISK_1/ /mnt/temp". (just make sure /mnt/temp exists).

By the way, are you able to deploy instance from template?


Comment actions Permalink
Avatar
Administrator
Avatar

Hi,

Thanks for taking the time to respond to this post. When I manually mount the storage directly to the Xenserver there is no problem doing so. When I use the Centos 5.6 template to deploy there isn't an issues with creating an instance. Its when I try to create a instance with the templates I generated/imported into CloudPlatform, then I get the errors stated in the first post. I can get around this by updating the cloud database, but would like to get away from doing this in the future.


Edward Valencia MEMBERS
Comment actions Permalink
Avatar
Administrator
Avatar

Evalencia , mounting from XenServer is one thing but you also need to be able to mount if from your secondary storage VM. Can you double check if you can mount this NFS from SSVM?

Radek.


Radoslaw Smigielski CITRIX EMPLOYEES
Comment actions Permalink

Top Contributors