Remi Bergsma's blog

CloudMonkey is an easy way to interact with the CloudStack API without the need to do any code writing. Think of it like a way to “query” the CloudStack infrastructure. Very useful in day-to-day operations to get a “quick look” at things.

Version 5.3 is on its way, and recently I had a look at it. I used my MacBook Pro for this, and it came with some challenges to get it working. So, I’m writing down here how to get CloudMonkey 5.3 working on either MacOSX Mavericks (10.9) or the new Yosemite (10.10).

Installing CloudMonkey from source
Since I wanted to look at the latest (not yet released) version I used the source from git. If you want to use the latest stable version, please skip to the next section where I discuss installing using PIP.

Let’s install from source:

git clone https://github.com/apache/cloudstack-cloudmonkey.git

Something like this should happen:

Cloning into 'cloudstack-cloudmonkey'...
remote: Counting objects: 782, done.
remote: Total 782 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (782/782), 236.67 KiB | 359.00 KiB/s, done.
Resolving deltas: 100% (427/427), done.
Checking connectivity... done.

If it doesn’t work, you might need to install Git or install the Command Line Developer tools. Just follow the on-screen instructions and you’ll be fine.

Next, compile the source:

make build

It’ll give some output, and as long as it doesn’t error out you’re fine. Then install using sudo:

sudo make install

CloudMonkey has now been installed to /usr/local/bin.

Installing CloudMonkey from PIP
Alternatively, you can use PIP to install. PIP is a tool to easy install Python software and there’s a PIP for CloudMonkey as well. At the time of writing, there is a 5.2 version that you can install.

First install to the latest version of PIP:

sudo easy_install --upgrade pip

If CloudMonkey is already installed, uninstall it first (your config is saved):

sudo pip uninstall cloudmonkey

Then install it again:

sudo pip install cloudmonkey

Running CloudMonkey

To run it, just type:

cloudmonkey

 CloudMonkey should then welcome you:

On Mac OSX Yosemite this worked out-of-the box, although on Mavericks it did not:

Remis-MBP:git remi$ cloudmonkey 
Traceback (most recent call last):
  File "/usr/local/bin/cloudmonkey", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2603, in <module>
    working_set.require(__requires__)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 666, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
    raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: requests
Remis-MBP:git remi$

There’s a lot written on this online, but most stuff did not work for me. This did the trick for me:

sudo curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | python

It (re)installs ‘setuptools’ and that made it work.

Configuring CloudMonkey

Before CloudMonkey will do anything useful, you need to point it to a CloudStack API endpoint. I’m using my DevCloud environment here. The API keys can be generated in the UI:

If no keys are displayed, click the icon on the right to generate keys. Then setup CloudMonkey:

set apikey whMTYFZh3n7C4M8VCSpwEhpqZjYkzhYTufcpaLPH9hInYGTx4fOnrJ3dgL-3AZC_STMBUeTFQgqlETPEile4_A
set secretkey 9Z0S5-ryeoCworyp2x_tuhw5E4bAJ4JTRrpNaftTiAl488q5rvUt8_pG7LxAeg3m_VY-AafXQj-tVhkn9tFv1Q
set url http://127.0.0.1:8080/client/api

Make sure to copy your own keys, as the ones here will only work on my local test CloudStack environment.

Finally, test the connection by syncing the API’s. Run this inside CloudMonkey:

sync

CloudMonkey responds:

466 APIs discovered and cached

Depending on your permissions (root, admin, user) and version of CloudStack, you’ll see more or less APIs being discovered.

A cool new (experimental) feature is tab completion, aka paramcompletion. Enable it like (run inside CloudMonkey):

set paramcompletion true

You’ll then be able to use the TAB key to auto-complete the UUIDs. Let me show you with an example:

I typed:

li<tab> vi<tab>m<tab> id<tab><tab>aa<tab><enter>

This auto-completes to the line you see in the image, and it displays the result as well.

Conclusion

CloudMonkey is a great tool to query the CloudStack infrastructure. Setting it up is not hard as we’ve seen. Justplay with it, and you’ll discover a great number of features. If you find bugs, be sure to contact Rohid Yadav, the author of CloudMonkey. The documentation can be found here.

Have fun with CloudMonkey!

Sed is very powerful, I use it a lot on Linux servers I manage. Today I was working on a local git repository on Mac OSX Mountain Lion when I run into some trouble.

Usually, to replace text in a file with new text I run:

sed -i 's/Find this text/Replace with this/' file_to_replace_in.txt

While this works on Linux, it does not on Mac OSX:

sed: -i may not be used with stdin

The manpage on OSX says:

Edit files in-place, saving backups with the specified extension.
If a zero-length extension is given, no backup will be saved.

Aha, it wants to save a backup file. So I changed my command to:

sed -i '.bak' 's/Find this text/Replace with this/' file_to_replace_in.txt

This works, although it leaves a backup file ‘file_to_replace_in.txt.bak’ behind. This is great if you’re not sure, but can be annoying as well. To stop it making backups you specify an empty extension, like so:

sed -i '' 's/Find this text/Replace with this/' file_to_replace_in.txt

This allows me to quickly find & replace again, like when working on Linux 🙂

Last week I wrote a small blog about OpenVPN on OSX Mountain Lion using Tunnelblick. I managed to get things to work using the latest Tunnelblick beta. But after a week working with it, I’m not too happy how it works right now. The two issues I have are:

1. DNS servers do not always get set properly, it feels unstable to me. I’ve seen /etc/resolv.conf with the right content, but still the old DNS servers were used by Mountain Lion. This is annoying: while connected to the vpn, hosts behind it do not resolve because the DNS server advertised by the OpenVPN server is not being used. This happened a few times last week, although most of the time it works ok. What goes wrong more often is the opposite: when disconnecting the connection, the OpenVPN advertised DNS server is still being used; but since we’re disconnected, it doesn’t resolve anymore and so nothing works. I’ve then to manually restore the right DNS servers. No fun, but I accepted it for now. I thought there’d be an update soon enough that’d solve this.

2. Problem two is more annoying: the connection is not stable, it’s slow and when working in a shell it is annoying to wait for the cursor to move. It frustrates me. I’d already blamed the guys in the office for heavy downloading only to discover it was Tunnelblick that was the problem. Sorry guys 😉 Connections to public hosts are fine by the way; I wasn’t really able to pinpoint this issue. It kept me from working effectively. If somebody knows what to do to get this working properly, let me know!

So I thought it’d be wise to look for an alternative, and I came across Viscosity. I just wanted to compare and see if it has the same issues. Don’t get me wrong: I always prefer open source software, but it does have to work. Tunnelblick and Mountain Lion clearly doesn’t work for me. Viscosity is $9 for a license, that sounds ok to me.

Downloading and installing is easy, like any other OSX app you open the dmg and drag the application to the application folder. That’s it. What surprised me was that it has an import feature that imports from Tunnelblick. One click, that’s all..

Now that’s cool. From the upper menu, you can now connect. I just works, a few seconds later I’m connected and.. DNS is handled properly. I tried connecting, disconnecting and all but can’t find any problem. It uses the OpenVPN specified DNS servers while connected, and the DHCP specified DNS servers while disconnected. Just like when I was running Tunnelblick on Lion. Also, I haven’t seen the slow connection issue. It’s a bit early to tell if that is gone for good. Since I have 30 days to try Viscosity, I’ll soon enough know 🙂

For me this is a fast en good solution. I hope Tunnelblick is able to sort out what goes wrong with Mountain Lion. Otherwise Viscosity probably has a lot of new users to welcome in the coming months. Kudos to the Sparklabs team!

Update: I did go back to Tunnelblick a few times but had no lock and didn’t want to spend more time on it. Then I bought Viscosity that just works and has been stable from the start.

I’ve upgraded my 3 year old MBP with a fast SSD. I decided to install from scratch instead of migrating or cloning the old disk. The good thing is that everything is now fast and clean. The downside is that all customisations I did over the past years are also gone.

One thing I customised was shortcuts. I forgot how easy this can be done in OSX, so I’ll describe it below so others can benefit as well.

For example, I want the shortcuts in Terminal for Next and Previous tab to be CMD+right arrow, and CMD+left arrow. In Terminal you can see the default shortcuts for this in the menu:

It looks ok, but you cannot directly press ‘}’ on most keyboards; It’s SHIFT+], so this makes the default shortcut CMD+SHIFT+]. I prefer a shortcut with just two keys. To change these you’d go to:

System Preferences | Keyboard | Keyboard Shortcuts

From the left, choose ‘Application Shortcuts’ and then click the + icon to add a new shortcut. First, select the application you want to change the shortcut for, ‘Terminal.app’ in this example. Then, specify the exact menu title for the command you’d like to change the shortcut for. As you can see above, you’d enter ‘Select Next Tab’ to change that command. Repeat for ‘Select Previous Tab’. Just press the shortcut and it will appear in the box. Afterwards it should look like this in the keyboard preferences pane:

Now, go back to the Terminal application and have a look at the same menu as before. It should now list the shortcut you just added.

Try changing tabs by pressing CMD+left/right arrow.. it works!

One thing to note, it that the shortcuts I used in this example, by default belong to changing windows instead of tabs. When you want to change windows in Terminal using a shortcut, you’ll need to add new shortcuts for these as well. Alternatively, you use the CMS+` shortcut which still cycles through your windows, it just cannot go back. For me this works ok as this works in any application.

Changing shortcuts in Mac OSX is easy and allows you to make them work exactly as you prefer. I’ve done this in Mountain Lion, but I believe this can be done in Lion as well.

I just upgraded to OSX Mountain Lion and had some trouble this morning getting my OpenVPN connection (using Tunnelblick) to work. It would connect, but the DNS resolving was unstable. I managed to get it to work on the latest stable by changing the DNS-server setting to ‘set nameserver 3.0b10’ but then internet traffic would not work. The best solution for me at the moment is to upgrade to Beta release 3.3beta16 3.3beta18.

After upgrading, everything works as expected!

More info:

1. Mountain Lion DNS issues with Tunnelblick
2. Discussion the issue

Update: see this blog I wrote as well (experience after one week)

Update 2: TunnelBlick 3.3 beta 18 has just been released and the release notes state it has fixed more Mountain Lion bugs.

I’ve had some trouble when using the Mac OSX Terminal app for some time now. Until today, it just gave me some annoying warnings from time to time. Like when installing an application with apt-get in Debian:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = “UTF-8”,
LANG = “en_US.UTF-8”
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

It did work, so nothing too serious. I’ve also found applications, like iotop for example, that refuses to start when LC_ALL was unset. But a quick

EXPORT LC_ALL=$LANG

made the application start, so I didn’t take the time to investigate it further. Today I run into a more serious issue that cost me quite some time to figure out.

I had stopped the pure-ftpd deamon to do some maintenance and then started it again. It did start without error, but connecting failed:

server:~# ftp ftp.server.nl
Connected to ftp.server.nl.
perl: warning: Setting locale failed.
ftp>

Nothing had changed in the ftp configuration. After some debugging and trial & error, I found out that when I started the deamon from within a shell on Ubuntu it worked, but when I started it within a shell on my MacBook, it didn’t.

When looking at the locales I found:

server:~# locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE=UTF-8
LC_NUMERIC=”en_US.UTF-8″
LC_TIME=”en_US.UTF-8″
LC_COLLATE=”en_US.UTF-8″
LC_MONETARY=”en_US.UTF-8″
LC_MESSAGES=”en_US.UTF-8″
LC_PAPER=”en_US.UTF-8″
LC_NAME=”en_US.UTF-8″
LC_ADDRESS=”en_US.UTF-8″
LC_TELEPHONE=”en_US.UTF-8″
LC_MEASUREMENT=”en_US.UTF-8″
LC_IDENTIFICATION=”en_US.UTF-8″
LC_ALL=

Notice the two errors at the top. I talked to a colleague about this and he suggested looking at the Terminal app settings. There I found a setting called “Set locale environment variables on startup” which was activated. The setting is located in Preferences | Settings | Advanced. I’ve unchecked the button now as you can see in this screenshot:

When closing the Terminal app, and reopening it again, I tried again:

server:~# locale
LANG=en_US.UTF-8
LC_CTYPE=”en_US.UTF-8″
LC_NUMERIC=”en_US.UTF-8″
LC_TIME=”en_US.UTF-8″
LC_COLLATE=”en_US.UTF-8″
LC_MONETARY=”en_US.UTF-8″
LC_MESSAGES=”en_US.UTF-8″
LC_PAPER=”en_US.UTF-8″
LC_NAME=”en_US.UTF-8″
LC_ADDRESS=”en_US.UTF-8″
LC_TELEPHONE=”en_US.UTF-8″
LC_MEASUREMENT=”en_US.UTF-8″
LC_IDENTIFICATION=”en_US.UTF-8″
LC_ALL=

No more errors! I tried restarting the pure-ftpd deamon from my Terminal app and it now works as expected. Even the warnings and errors when installing applications in Debian (apt-get) are gone. In fact, it seems this is the way it is supposed to work.

Glad I’ve fixed this 🙂

Update: As Reza mentions in the comments, it’s also possible to fix this problem on the server side. This is the best way to go if you want to fix this for your users. Thanks Reza!

Today I came across a nice post by Tod Werth, who created a nice theme for the OSX Terminal program called IR_Black.

All you have to do is download his schema and open in it Terminal. Then tweak the colors a bit to fully meet our needs.

Last step: enable colors in your profile:

vim ~/.bash_profile

Add this line:

export CLICOLOR=1;

When you open a  new window and enter a simple ‘ls’ command, it looks like this:

So that is pretty cool 🙂 Thanks Todd!

Update:
I found that using these setting brought some trouble when working in vim and nano. Changing the terminal from “xterm-256color” to “xterm-color” fixed that for me.

Adding a route manually can be necessary sometimes. When on Linux, I know the command by head:

sudo route add -net 10.67.0.0/16 gw 192.168.120.254

On the Mac the command is similar, but a bit different 🙂 Just as a note to myself and anyone else interested:

sudo route -n add -net 10.67.0.0/16  192.168.120.254

This sets up a route to the 10.67.0.0/16 net through gateway 192.168.120.254. First one on Linux, second one on Mac OSX.

I’m working in Terminal a lot. Lately I more and more use my MacBook Pro instead of my Ubuntu desktop. I’d to overcome some small issues. One was the missing Home-key, which allows you to jump all the way to the left on the current line.

Although the Mac doesn’t have a Home key, its function is available by pressing Fn+Shift+Left Arrow. And even better: Fn+Shift+Right Arrow jumps all the way to the right.

I feel right at home! 🙂

By default, the Alt-key doesn’t work in Mac OSX-Terminal. Today I found a simple solution for this problem:

Terminal > Preferences > Settings > Keyboard
Just check ‘Use option as meta key’ at the bottom and you’re good to go 🙂