Depending on who you ask, DevOps is either the best thing to come along since sliced bread or the biggest load of hooey ever foisted on the IT sector.

But while it’s understandable to have an allergic reaction to yet another cool technology that will Save Us All and still have time to bake cookies (with sprinkles), there is some real substance within the notion of DevOps … provided you can get past the hype.

In full-on hype mode, DevOps seems to suggest that development and administrative teams are somehow at odds and that DevOps will part the waters that lie between these two groups and align both groups around common business goals.

Putting that aside, the genuine benefits of DevOps are typically a bit more mundane, yet no less useful: You have this thing called cloud computing, and to really make use of it properly, the machine resources in the environment have to be able to be deployed and otherwise administered on an automatic basis. This automated management is a big part of what cloud computing really is: A collection of virtual machines without such controls is just a virtual datacenter.

This is where the need for DevOps really comes to the fore: Application developers not only have to create apps for distributed systems, they also have to put code together for distributed systems that scale one way or the other. This is another buzzword: elastic. But it’s a good descriptor, nonetheless.

Cloud computing is not the only reason the DevOps trend has risen, mind you: The need to coordinate the functionality of any IT environment’s hardware and the software that runs on it has always been around. But what distributed systems and now elastic systems have done is made the need for such coordination a necessity rather than a “nice to have” option.

The pain of this necessity arrives in this simple fact: Programmers might be great programmers, but not so good at understanding what’s involved in administering systems. Conversely, administrators might know their machines cold, but not intuitively understand what apps really need to function at their best.

DevOps is the coupling of these two weaknesses to create tools that make up for the gaps in each group’s knowledge. And the tools being built these days are pretty exciting to see.

The Charms of juju

With the release of juju in Ubuntu 11.10, Canonical not only rebranded its former Ensemble service, it also streamlined a big component of its “service orchestration” platform.

Juju is one of those tools that takes a bit to explain, but when the light bulb goes off, you just “get it.” Perhaps it’s the arcane verbiage that’s floating around this tool: the magic of juju and the use of “charms.” But if you know a little bit about the history of folklore and magic, these descriptions immediately become less cute and more useful.

Essentially, juju is an automated service deployment tool that enables developers and administrators to spin up any sort of pre-configured node of any sort of service. This pre-configuration is done with the use of charms. Like the intended use of the word in magical circles, a charm is a way to “store” a spell for later use without having to lug around spell books and eyes of newt. In magical universes, charms are like the Swiss army knives of prestidigitation.

In the context of juju, that’s exactly what charms are, too. If, say, someone wanted to spin up a master node of a Hadoop server, they could use Bazaar to download the appropriate juju charm, connect juju to the environment where the node will be deployed (such as an Amazon EC2 cloud), then issue two commands on the CLI:

juju bootstrap
juju deploy hadoop-master

And that’s it. Everything needed to run the master node of Hadoop on that system you pointed it to will be installed, configured, and run just as fast as that environment can go. The Canonical team describes juju as apt for the cloud, and functionally, it performs in just that way.

If any of this looks familiar to you, you might be thinking of other service-in-a-can systems that would also seem to do the same thing. The one that came to mind for me was Bitnami, BitRock’s awesome tool to build complete systems like this on native, virtual, and cloud-based systems. What is new here that Bitnami doesn’t already do?

Turns out, quite a bit. If you’re not familiar with Hadoop as a distributed data storage system, it typically needs a single master node and as many slave nodes as you need to get the job done. So, you need a Hadoop slave-node charm and this command

juju deploy hadoop-slave

Again, not much new here. Until you get to the really cool stuff in juju:

juju add-relation hadoop-master hadoop-slave

This, really, is what makes juju a service orchestration tool. Like apt for local software and third-party tools like Bitnami for full-stack installs, juju does a great job pulling tools together and installing them on the fly. But what it also does is build relations between these nodes so they can immediately work together. That’s the big differentiator of juju.

With this capability to run services on demand and then connect those services with a single command, juju becomes a very powerful way for developers to start building operator functionality into their applications. And scaling is even easier. Need more nodes? Then a command like this will get you another:

juju add-unit hadoop-slave

Need a bunch more nodes fast? Then all you need is:

juju add-unit n=50 hadoop-slave

Besides its use for physical provisioning, juju also has potential for launching local developer systems and public cloud and hybrid cloud administration. What’s key to note here is that throughout juju, administrators and programmers are not looking at the machine instances running these services – they’re handling the services themselves. All of the configuration work is pre-built into the charms, which makes it a lot easier to think in these terms.

Currently, juju has 88 charms available for use, with more being added all of the time. Canonical is encouraging administrators and developers to build their own charms and have put together some good tutorials on how to build and contribute charms.

Such service orchestration would seem to be replacing configuration management tools such as Puppet, but juju team members have emphasized that this is not really the goal. In fact, charms can be written to delegate configuration tools to something like Puppet, if Puppet scripts have already been built.

Although juju was first made available in Ubuntu 11.10, the tool is expected to hit full maturity in the upcoming 12.04 release. Expect a big push on the part of Canonical touting its cloud orchestration services at that time, with juju very much a centerpiece.

Nodeable Events for Admins

Juju represents a powerful way for developers to connect to systems and services on an operational level, but what about the reverse? With distributed apps performing so much auto-configuration and generating so much process data and data about the performance of the app itself, how can administrators keep abreast of the flow of information?

This is the question Dave Rosenberg, CEO of Nodeable asked himself. Systems administrators were dealing with a flood of new information from distributed and elastic systems that were not only going up and down on demand but also self-activating images, reporting poor latency and storage limit ceilings, and having new code committed at any given time.

Monitoring tools such as Nagios and Splunk can help track such change-of-state events, but not necessarily in real time.

Rosenberg looked at this flood of information and came up with an elegant response. Thinking of another technology sector that enables people to manage one-to-many relationships, Rosenberg only had to look as far as Twitter and Facebook to find his answer.

For Rosenberg, the capability to bring in information from multiple data sources was the real value-add of social media services, and so it was a good fit to apply to IT administration. Thus was born Nodeable.

Initially, it’s easy to dismiss Nodeable as “twitters from your server” because that’s only the cursory view of what Nodeable does. Nodeable is actually best at aggregating data into coherent messages that can be delivered to whomever or whatever you need.

“Whomever” can include admins, programmers who need to see fault reports, and even IT managers who just need to keep on top of things. “Whatever” can be any system admin or any monitoring tool you need to track the aggregated Nodeable data.

Interestingly, even though Nodeable has potential as a report-and-react tool, Rosenberg is finding that, for now, customers are really enjoying just the capability to watch their systems and see how they perform at different times and in various scenarios.

“They want to know what’s changed,” Rosenberg said of Nodeable’s current user base.

Ultimately, he added, users of the hosted Nodeable service will be able to react to the data that’s being sent to them, introducing a channel for systems administration through the Nodeable framework.

“We’re building an information network for systems and services,” Rosenberg said. Such a network will eventually enable a two-way communication line between operations and their machines and developers and their applications – a line that won’t be flooded with a cacophony of extraneous data.

Wrapping Up

The open source landscape is a perfect environment to join the needs of development and operations together, as these two new tools clearly demonstrate. DevOps might fade as a fad, but the needs of cloud and distributed computing makes the relationship between software and hardware all that more important to couple, so that building and maintaining such systems doesn’t get out of control.

DevOps tools like juju and Nodeable can help do just that.

The cookies with sprinkles, sadly, are optional.