0

I recently installed Openshift Origin on a server (clean installation, I am not using the VM). I can create small size NodeJS applications, but when I specify the gear size medium (either through rhc or the web administration interface) I get this error : Unable to complete the requested operation due to: No nodes available. The user I am doing this with as access to 100 nodes and all node sizes. Anybody knows what might be happening here?

Besides, the medium gear size is needed to accommodate a KeystoneJS app.

--- EDIT ---

I was unable to fix the server, so I started the Openshift installation from scratch on a new CentOS install. I configured an all-in-one server with only one medium node within the default district. But after installation, running oo-diagnostics gives me these errors:

WARN: test_node_profiles_districts_from_broker The following gear profile(s) are configured but not provided by any node hosts: medium Attempts to create apps using these gear profiles will fail. Please fix the settings in /etc/openshift/broker.conf or add node hosts accordingly.

WARN: block in test_node_profiles_districts_from_broker There are no node hosts in district 'Default' FAIL: test_node_profiles_districts_from_broker Default gear profile 'medium' has no active node hosts supplying it in any district. Attempts to create apps without specifying gear size may fail. Please add active node hosts to a district with profile 'medium' using oo-admin-ctl-district or fix the settings in /etc/openshift/broker.conf

FAIL: test_node_profiles_districts_from_broker The following gear profile(s) are configured: small, medium but not provided by any active district hosts. Attempts to create apps using these gears may fail. Please add districts / node hosts with oo-admin-ctl-district or fix the settings in /etc/openshift/broker.conf

WARN: block in test_node_profiles_districts_from_broker Node host master.os.bonqoeur.ca with profile 'small' is not a member of any district. Please add it to a district with oo-admin-ctl-district.

Also, on this install I cannot create medium sized apps, but small apps works fine (even though no node is supposed to be available)... Anyone has a clue?

2

This issue could go many different ways, so I'll try to cover as many as I can here and hope that it moves you in the right direction.

First, I would ensure that you have a medium node profile (district) with available nodes. You can verify your districts by running oo-admin-ctl-district on your broker system.

You will be presented with output like this, that contains a block for each district.

{"_id"=>"55131197ecdd5c0555000001",
 "active_servers_size"=>2,
 "available_capacity"=>6000,
 "available_uids"=>"<6000 uids hidden>",
 "created_at"=>2015-03-25 19:50:47 UTC,
 "gear_size"=>"medium",
 "max_capacity"=>6000,
 "max_uid"=>6999,
 "name"=>"medium",
 "platform"=>"linux",
 "servers"=>
  [{"_id"=>"551318bbecdd5c0555000001",
    "active"=>true,
    "name"=>"node0.example.com",
    "unresponsive"=>false},
   {"_id"=>"551318d3ecdd5cd555000001",
    "active"=>true,
    "name"=>"node1.example.com",
    "unresponsive"=>false}],
 "updated_at"=>2015-03-25 19:50:47 UTC,
 "uuid"=>"55131197ecdd5c0555000001"}

Notice in the output where it shows active=true and unresponsive=false for the nodes within the district. This indicates that a node is healthy and responsive.


Mcollective should show that all your nodes are pingable when you run oo-mco ping from your broker system.

node0.example.com    time=220.43 ms
node1.example.com    time=228.63 ms

If you can't ping your nodes, you may have an issue with iptables or possibly mcollective configuration. Make sure that your nodes can communicate with your msgserver on port 61613. Also ensure that your node systems are correctly configured to communicate with the msgserver as the correct user, etc.

On node hosts, this configuration lives in /opt/rh/ruby193/root/etc/mcollective/server.cfg.


Next, assuming that your entire installation is contained on one system, you may only have a small node profile available (the one system). If this is true, you can create an additional node as well as a district/node profile for that node. This will require separate infrastructure with different configuration in /etc/openshift/resource_limits.conf.

To create a new district,

oo-admin-ctl-district -c create -n medium -p medium
oo-admin-ctl-district -c add-node -n medium -i yournode1.example.com

More information on district creation and population for origin is available here: http://docs.openshift.org/origin-m4/oo_administration_guide.html#creating-and-populating-districts

| improve this answer | |
  • Thanks! I could create the medium district using "oo-admin-ctl-district -c create". But now I get an error upon node creation: >>oo-admin-ctl-district -c add-node -n medium -i node.example.com /usr/sbin/oo-admin-ctl-district:215:in block in <main>': undefined method casecmp' for nil:NilClass (NoMethodError) from /usr/sbin/oo-admin-ctl-district:178:in block in collate_errors' from /usr/sbin/oo-admin-ctl-district:176:in each' from /usr/sbin/oo-admin-ctl-district:176:in collate_errors' from /usr/sbin/oo-admin-ctl-district:213:in <main>' – Olivier Apr 12 '15 at 11:16
  • @Olivier This error looks to be caused by nothing being returned from this line of /usr/sbin/oo-admin-ctl-district: node_platform = OpenShift::MCollectiveApplicationContainerProxy.rpc_get_fact("kernel", server) {|_, _|}. In the version of the script I have, node_platform is checked to be nil and provides a helpful message: Cannot connect to node. Is ruby193-mcollective running on your node host? If you open the script up, do you see similar around line 213? casecmp is only used once so I'm pretty sure this is the problem. – abutcher Apr 13 '15 at 0:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.