Using manual environment for below sequence.
Note that the below sequence assumes that you utilize virsh snapshots. It is highly recommended to
setup the nodes and then snapshot each of them so you can go back to clean point easily. For the IBM
POC we had (using the openstack 40 bundle)
- VM1 that ran the MAAS/juju. This was on separate compute node.
This VM we considered either MAAS deployer or jujum deployer server.
Snapshot of this was made where MAAS/juju installed where no environment built.
- VM2, VM2, VM3 and VM4 running ubuntu 14.04 on one node
Snapshot of all these VMs built with network setup to talk to each other and NO jujud client on end points.
This checklist indicates how to rebuild quickly if the VM's were provisioned by MAAS and then you
need to just rebuild via juju. The MAAS is used to provision. The juju is used via the manual envionment
to provision all the software. If it failes or something gets messed up then you can clean up VMs and
then rebuild using the juju manual environment. This is really easy.
1 Clean up the endpoints
How to clean up busted end point?
How to resolve "ERROR machine is already provisioned" in manual provision set up?
rm -rf /var/lib/juju
rm -rf /etc/init/juju*
ps -ef | grep juju
kill the process machine using juju
How to I revert to a snapshot for a busted VM? if environment busts
http://kashyapc.com/2011/10/04/snapshotting-with-libvirt-for-qcow2-images/ see bottom of page
command example:
outside virsh
virsh snapshot-revert --domain ubuntovm --snapshotname 1441158838
in virsh
snapshot-revert --domain vm3 --snapshotname 1444163918
2. Check all endpoint VM have access to internet
This can be done various ways. We used below so we could have Firefox UI available if needed to the endpoints.
Start following on each VM:
x11vnc -create -forever -bg -env FD_TAG=my_xfce_1 -env FD_SESS=xfce -rfbport 5901
Note that at the time of this document you cannot use vnc for ppc64le. Use
x11vnc. You have to install the x11vnc prior to this command, not discussed here.
BSO (IBM) and firefox access:
Make sure you can access:
https://streams.canonical.com/juju/tools/releases/juju-1.24.6-trusty-ppc64el.tgz
3. Clean up the juju server
How do I uninstall and reinstall juju on deploy server due to issues?
You can clean up the juju deploy server completely as listed below or you can
short-cut and remove the /root/.juju/ files and start over again. Depends
on how clean you want to start. Cleaning jup /root/.juju/ and start over
is usually good enough. Note if you create proper snapshot of VM don't have to do this.
To remove completely:
sudo apt-get remove juju-core
sudo apt-get remove --auto-remove juju
sudo apt-get remove juju-quickstart
sudo apt-get remove juju-deployer
/root/.juju/ remove directory and all files directory under it
Example:
rm -rf /root/.juju/jclient
rm -rf /root/.juju/environments
rm -rf /root/.juju/ssh
rm -rf /root/.juju/.deployer-store-cache
Then reinstall packages:
sudo add-apt-repository ppa:juju/stable
sudo apt-get update
sudo apt-get install juju-core
sudo apt-get install juju-quickstart
sudo apt-get install juju-deployer
juju-quickstart -i
Note if you are testing probably want to put the installable bundle in another directory so you don't lose it. Suggestion is something like this:
/root/.juju2/bundles
4. Rebuild the environment (juju deployment server environment)
Just build the environment, mark it default and save it. Do not execute from juju-quickstart -i.
For the IBM POC we used MAAS as initial VM provisioning. Once that is done then you can create snapshots and after that onlly have to rebuild the VMs fron snapshots. If you do it this way the juju environment you use is the manual definition since the target endpointswere already provisioned.
5. Bootstrap environment
juju bootstrap --show-log --debug -e manual
If you are using juju first time or during testing then add --debug and --show-log because
it gives you good feeling of progression.
6. Add in all the other servers
The manual environment puts the 1st VM into the environment. You then use the juju add-machine command to add all the other VMs. For openstack 40 bundle you
need total of 4 machines so do the following (set with your ipaddresses obviously)
juju add-machine ssh:root@9.x.xx.xxx -show-log --debug
juju add-machine ssh:root@9.x.xx.xxx -show-log --debug
juju add-machine ssh:root@9.x.xx.xxx --show-log --debug
7. Deploy environment
For IBM POC we had to copy openstack40.yaml from charm store because
it had to be fixed to ppc64el architecture. We also made some other
changes to the yaml to make it work.
juju-deployer -c /root/.juju/bundles/openstack40.yaml -s 90 -d -v openstack40.yaml
The above openstack40.yaml has 3 changes:
1. Use of ppc64el
2. for openstack dashboard add option
debug: 'yes'
3. Passwords for mysql
root-password: PASSW0RD
sst-password: PASSW0RD
NOTE: Step 8 has parts that overlap this step, read step 7/8 together.
8. During deploy process fix lxc container config files
For this POC the lxc did not start on 1st two VMs. We did not debug
far enough. Not know if everyone else will hit this but we are listing
it here just in case others hit.
Deployed juju bundle OpenStack 40 has linux containers not started?
When deploying OpenStack 40 bundle the environment does not have
juju-trusty-lxc-template installed at environment build. When you start deploying
the juju-trusty-lxc-template is eventually installed. So this is suggested for the
first two machines in the juju status list. Have to fix them.
Edit /etc/lxc/default.conf and add
lxc.seccomp =
lxc.arch = ppc64le
During the deploy process watch directory /var/lib/lxc/
for creation of juju-trusty-lxc-template/ directory.
I used winscp to watch the file creation during deploy. No real good way
to do this so you have to watch the deployment proccess and check
for file every so often. When there the lxc exists, is pending and does
not get started till late in process. So you have time to fix the file.
Once directory exists then
vi /var/lib/lxc/juju-trusty-lxc-template/config and add:
lxc.seccomp =
lxc.arch = ppc64le
Make sure blank line at the end. If not then additional parameters
are added and they may be appended to the last line you added
incorrectly.
We did not find a good way to do this so above troubleshooting question
explains what to do while deploy is running.
9. mysql charm fails and requires passwords to be setup
See other askubunto for this problem mysql fails due to
"Unable to set password for Percona". See:
mysql/percona: Unable to set password for the Percona Server "root" user- ppc64el
Hints for debug and watching the deploy:
a) on juju deployer machine
Make sure you have this so you can check process.
When rebuilding environment in separate window, in /root/.juju/ issue
juju status > jujudeploylog10132015.txt
to periodically to save logs, change date in file
b) Suggestion is to have two windows open in SSH, one for the view of the
juju-deployer command and another in 1st VM in manual environment
doing tail in /var/log/juju/ tail -f ./all-machines.log
This allows you to observe the building of the openstack environment.
c) If using the openstack 40 bundle then the OpenStack horizon UI is on the
last VM. Checking juju status info tells you the lxc that horizon is on.
Use this: http:///horizon
