Here is the output of my "catalina.out" log when I click the "Launch Cloud Console" button. None of the other logs really have anything good in them in the /var/log/cloud/portal directory

##################################

DEBUG 02 Jul 2012 21:14:26,258:[http-8080-5:DispatcherServlet][] DispatcherServlet with name 'vmops-portal' processing GET request for [/portal/portal/users/cloud_login]
DEBUG 02 Jul 2012 21:14:26,258:[http-8080-5:DispatcherServlet][] Last-Modified value for [/portal/portal/users/cloud_login] is: -1
INFO 02 Jul 2012 21:14:26,258:[http-8080-5:UserContextInterceptor][] Before handling the request
DEBUG 02 Jul 2012 21:14:26,258:[http-8080-5:UserServiceImpl][] ###reloading user 1
DEBUG 02 Jul 2012 21:14:26,290:[http-8080-5:UserServiceImpl][] ###refreshing user 1
DEBUG 02 Jul 2012 21:14:26,293:[http-8080-5:UserServiceImpl][] ###reloading user 1
DEBUG 02 Jul 2012 21:14:26,294:[http-8080-5:UserServiceImpl][] ###refreshing user 1
DEBUG 02 Jul 2012 21:14:26,296:[http-8080-5:UserServiceImpl][] ###reloading user 1
DEBUG 02 Jul 2012 21:14:26,296:[http-8080-5:UserServiceImpl][] ###refreshing user 1
DEBUG 02 Jul 2012 21:14:26,313:[http-8080-5:AbstractUsersController][] ###Entering in login(tenantId,map) method @GET
GRANTED due to ACE: AccessControlEntryImpl[id: 1; granting: true; sid: PrincipalSid[root]; permission: BasePermission[...............................R=1]; auditSuccess: true; auditFailure: true]
INFO 02 Jul 2012 21:14:26,349:http-8080-5:CloudManagementServiceImpl][] Verifying cloud account for User [Email=ROOT@ROOT.ROOT, firstName=ROOT, lastName=USER, username=root
DEBUG 02 Jul 2012 21:14:26,349:[http-8080-5:DefaultCloudServiceImpl][] ###Entering in execute method
DEBUG 02 Jul 2012 21:14:26,363:[http-8080-5:DefaultCloudServiceImpl][] ###Exiting execute method
DEBUG 02 Jul 2012 21:14:26,364:[http-8080-5:DefaultCloudServiceImpl][] ###Entering in execute method
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping forprojectlimit as it is probably new.
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping forprojecttotal as it is probably new.
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping forprojectavailable as it is probably new.
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping fornetworklimit as it is probably new.
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping fornetworktotal as it is probably new.
WARN 02 Jul 2012 21:14:26,455:[http-8080-5:XStreamMarshaller][] ####### Missing field mapping fornetworkavailable as it is probably new.
DEBUG 02 Jul 2012 21:14:26,456:[http-8080-5:DefaultCloudServiceImpl][] ###Exiting execute method
DEBUG 02 Jul 2012 21:14:26,464:[http-8080-5:DefaultCloudServiceImpl][] ###Entering in getLoginUrl method
DEBUG 02 Jul 2012 21:14:26,464:[http-8080-5:DefaultCloudServiceImpl][] ###Entering in generateSignature method
DEBUG 02 Jul 2012 21:14:26,464:[http-8080-5:DefaultCloudServiceImpl][] ###Exiting generateSignature method
DEBUG 02 Jul 2012 21:14:26,464:[http-8080-5:DefaultCloudServiceImpl][] ###Exiting getLoginUrl method
DEBUG 02 Jul 2012 21:14:26,464:[http-8080-5:AbstractUsersController][] ###Exiting login(tenantId,map) method @GET
INFO 02 Jul 2012 21:14:26,464:[http-8080-5:UserContextInterceptor][] Inside UCI Call...C
GRANTED due to ACE: AccessControlEntryImpl[id: 1; granting: true; sid: PrincipalSid[root]; permission: BasePermission[...............................R=1]; auditSuccess: true; auditFailure: true]
DEBUG 02 Jul 2012 21:14:26,479:[http-8080-5:DispatcherServlet][] Rendering view [org.springframework.web.servlet.view.RedirectView: unnamed; URL http://172.17.7.206:8080/client/?username=admin&domainid=1&timestamp=1341263666464&signature=X1FREbh10EeV8zHpkXoshb5JDD8%3D&tz=5.50] in DispatcherServlet with name 'vmops-portal'
DEBUG 02 Jul 2012 21:14:26,479:[http-8080-5:DispatcherServlet][] Successfully completed request
INFO 02 Jul 2012 21:14:26,479:[http-8080-5:RequestHandledListener][] Request handled: success... flushing events generated , UserName: root
INFO 02 Jul 2012 21:14:26,479:[http-8080-5:RequestHandledListener][] Request handled: success... flushing events generated , UserName: root

Hi Timothy,

Noticed that you have different jdbc credentials for cloud and cloud_usage databases. Could you try this from CPBM server to confirm if the necessary access is granted:

mysql -ucloud -pusername -D cloud_usage -h 172.17.7.206
mysql -uusername -ppassword -D cloud -h 172.17.7.206

For the last issue you are facing, you will have to set up a proxy server to get root user's single sign-on to CloudStack working. See "Setting up a proxy server" at http://support.citrix.com/proddocs/topic/cloudportalbusiness-14/ccpb-install.html

Hi fatima,

I have the usage working, I just didn't redact the username for that part of the config so my database metrics are working with no issues :-)

I have a proxy setup (its on a separate server) according to that article. How do I point the CS and CPBM server at it so we can use it?

I see in the 1.3x documentation where it talks about sharing a host. It also talks about 3 host and 5 host install. For this purpose we have a 3 host install and the proxy is sitting there awaiting its use I suppose.

Any further info would be much appreciated.

In cloud.properties on CPBM server, please change the vmops.mgmt.server.public* properties to point to the proxy server. Also, when accessing CPBM, use the proxy IP/host instead of CPBM directly.

How about the CS server, any changes need to be made there? Do I now change the cloud.properties to the proxy server as well for the section on the cloudstack connection?

Sorry for the questions, its not well documented.

We are working on addressing the gaps in documentation. Thanks for your patience.

The vmops.mgmt.server.public are the only properties that need to be changed. This is under the section

# Proxy Server information

The other sections which are below # CloudStack Server information and database sections still need to refer to the CS server.

Please iron out those deficiencies :-)

Ok Here is what I have:

CPBM: 172.17.7.205
CS: 172.17.7.206
Proxy: 172.17.7.207

The Proxy Server is a BARE bones install of CentOS 6.2 and ONLY packages installed were:

yum install httpd mod_ssl

Once I installed that I created the following conf: /etc/httpd/conf.d/cloud.conf

I have only 1 line in the conf: "ProxyPass /portal http://172.17.7.205:8080/portal" Please note that I am not using SSL but bare bones HTML at this time.

I gracefully restarted HTTPD via: apachectl graceful

I edited my cloud.properties files to have the following settings:

################################################################################
# CloudStack Server information
################################################################################
vmops.mgmt.server.publicHost=172.17.7.207
vmops.mgmt.server.publicPort=8080
vmops.mgmt.server.publicProtocol=http

# Semicolon separated list of host:port pairs of management server instances
# Setting below normally uses the port 8096
vmops.mgmt.server.serverList=172.17.7.206:8096
# Setting below normally uses the port 8080
vmops.mgmt.server.nonAdminServerList=172.17.7.206:8080

################################

Please note the 172.17.7.207 (Proxy) address that I placed in there.

I did a "service cloud-portal restart" to re-initialize the CPBM and I can confirm I can navigate to the CPBM web interface on 8080 under /portal.

When I point at the proxy via:

http://172.17.7.207
http://172.17.7.207:8080
http://172.17.7.207:8080/portal

I get no pages to be displayed. However when I got to:

http://172.17.7.205:8080/portal

I get to the CPBM portal with no issues.

What am I missing that your explanation or documentation hasn't covered?

Thanks in advance!

Hi Tim,

On the proxy server at /etc/httpd/conf.d/cloud.conf:
Add "ProxyPass /client http://172.17.7.206:8080/client"
Then restart apache.

Change vmops.mgmt.server.publicPort=8080 to 80

To access CPBM from the web browser, use http://172.17.7.207/portal/

Also, if iptables is running on 172.17.7.207, make sure to open port 80.

Again, we are working on fixing the docs to add some of these missing steps.

Thanks,
Fatima

Hi Fatima,

Thanks for the response. Unfortunately that does not work either. What I am puzzled about is that you have my proxy only redirecting the CS (172.17.7.206) and not my CPBM (172.17.7.205).

So when I change the cloud.conf to the CS address and use /client instead of /portal (although the 1.4 implementation instructions clearly denote to use the /portal not /client) when I browser to the http://172.17.7.207/portal I get (not surprisingly) "the requested URL /portal was not found on this server".

So I think to myself, well maybe if I access the CPBM server directly it will allow me to go backwards through the proxy connecting to CS from CPBM. So I log into: http://172.17.7.205:8080/portal and try that.

It yields the following result:

It redirects to http://172.17.207/client/?username=admin........ Which gives an error of: "Service Temporarily Unavailable".

I have no firewalls running on CS, CPBM or the proxy just in case that was an issue. I confirmed this with a NMAP scan and all ports are open and ready.

I think I am getting closer here, but still the same results as yesterday and no change in status.

Thanks for your reply.

I did not mean for you to replace the line you already had in cloud.conf to redirect CPBM. You are rightly puzzled and would need the directives for both /portal and /client. I was simply asking that you add the missing line for /client.

Once again, cloud.conf should have these two lines:

ProxyPass /portal http://172.17.7.205:8080/portal
ProxyPass /client http://172.17.7.206:8080/client

This is detailed in our 1.3 install docs, but is missing the directive for /client in 1.4. We have noted down these issues.

Thanks,
Fatima

That does make a lot more sense. But your 1.4 Instllation instructions only state:

Create the file /etc/httpd/conf.d/cloud.conf, and add proxying (and, if needed, load balancing) directives.
For a three-node installation where Apache, CloudPortal Business Manager, and CloudStack all run on separate nodes, use rules like the following. Replace CloudPortalNode and CloudStackNode with the private hostname or IP of your own machines.

When SSL is enabled

ProxyPass /portal ajp://CloudPortalNode:20410/portal
When SSL is not enabled

ProxyPass /portal http://CloudPortalNode:8080/portal

Then it goes into some CAS installation steps and never states to add the CS side. I did however locate this in the 1.3 documentation.

Ok now I have a my cloud.conf file with:

ProxyPass /portal http://172.17.7.205:8080/portal
ProxyPass /client http://172.17.7.206:8080/client

I still get the "Service is temporarily unavailable"

Here is what my error logs are saying:

[Tue Jul 03 17:40:01 2012] [error] (13)Permission denied: proxy: HTTP: attempt to connect to 172.17.7.205:8080 (172.17.7.205) failed
[Tue Jul 03 17:40:01 2012] [error] ap_proxy_connect_backend disabling worker for (172.17.7.205)

That's very interesting. I am googling now to see if I can find what the issue is. Keep in mind that I have a bare bones install, following the instructions to a T (except whats not documented of course) and still unable to proxy.

We are close now so we should be able to get the correct answers now with a bit of research.

And bingo! fixed the issue by adding:

setsebool -P httpd_can_network_connect 1

I then restarted the services and off we went. Although I am seeing an interesting error now when logging in. It never allows the login to occur. I should have that fixed in a minute...

Keep in mind this again is a bare bones out of the box install of CentOS 6.2. Enjoy the pic!

Hi Fatima,

Ok we have access to the system via the proxy but since in our test lab we are using IP's and not a domain we are getting the AJAX blocked by the OWASP CSRFGuard

https://www.owasp.org/index.php/CSRFGuard_3_Token_Injection

I see we can edit the Owasp.CsrfGuard.jar file to take the "domain-strict" out.

Its obviously easier to configure a DNS entry for the proxy and use FQDN's. Is there something I need to add to CPBM in order to allow that specific domain?

I just added an FQDN and I am still getting issues with the AJAX security and it complaining about the domain.

I do not have this issue when I dont go through the proxy.

I have tried:

portal.domain.lab
proxy.domain.lab (the actual name of the proxy server)

When I auth through the proxy it fails to a page:

http://portal.domain.lab/portal/j_spring_security_check

When I click "Return to homepage" it returns me to the CPBM server IP directly and not the proxy to log back in. Is this behaviour normal and expected?

Checking through the documentation again to make sure I didn't miss something about inputting a domain for security.

Any Idea on the OWASP CSRFGuard issue?

I ensured I added the IP of the proxy to the proxy.server.ip.list. I reboot all the servers and still get an security issue that prevents me from logging in.

I see nowhere in the docs for 1.3 or 1.4 where I can specify a domain, ip or anything that will allow for this. I am not using CAS authentication (oy that next) so there shouldn't be any issues.

I went through the configs and replaced all the IP's with FQDN's.....

Any clue as to why I am getting the attached error when going through the proxy?

Hi Tim,

Could you try using AJP instead of HTTP in the ProxyPass
Apache directives:

ProxyPass /portal ajp://172.17.7.205:20410/portal
ProxyPass /client ajp://172.17.7.206:20400/client

That may have been true in 1.3 but not in 1.4. As a matter of fact your docs state that if your not using SSL, then you have to use HTTP. To underscore this here is my error logs for the proxy:

WITH FQDN:
[Tue Jul 03 19:16:19 2012] [error] (70014)End of file found: ajp_ilink_receive() can't receive header
[Tue Jul 03 19:16:19 2012] [error] ajp_read_header: ajp_ilink_receive failed
[Tue Jul 03 19:16:19 2012] [error] (120006)APR does not understand this error code: proxy: read response failed from (null) (virgash-cpbm.sarum.lab)

With IP:
[Tue Jul 03 19:17:58 2012] [error] (70014)End of file found: ajp_ilink_receive() can't receive header
[Tue Jul 03 19:17:58 2012] [error] ajp_read_header: ajp_ilink_receive failed
[Tue Jul 03 19:17:58 2012] [error] (120006)APR does not understand this error code: proxy: read response failed from (null) (172.17.7.205)

Based on my research with OWASP CSRFGuard:

Checking the HTTP Referer Request Header is generally not an acceptable solution for three primary reasons: first, open redirects allow for GET-based CSRF attacks that originate within the accepted domain, second, header injection vulnerabilities allow attackers to set headers such as the Referer header therefore bypassing this protection, and thirdly, sometimes the Referer header is blocked by outgoing proxies and firewalls leading to usability issues

This is right off their website:
https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet

Not that it helps us fix the issue. The proxy is passing me to the CPBM portal. It thinks I am a cross site scripting attack. Technically that's true since we have provided nothing to the CPBM portal running the security that the proxy is a good place to come from.

We need to let the CPBM server somehow know we are good guys coming from a good place :-(