And at the end of it I was left with a slightly nervous feeling that in advocating the rise of DevOps, we run the risk of removing one archaic and broken system, only to replace it with more of the same.

The worrying thing was the hallway comments from people who were attending the event in order to formulate a plan to “implement a DevOps team” within their organization. That entire aim missed the point of DevOps as an organizational trend. I spent 45 minutes or so talking about how broken current approaches are – siloed teams, each with differing motivations and areas of focus doesn’t deliver consistency. Neither does the fact that these teams are generally talking in different languages, at cross purposes and with wildly different priorities.

To simply rip this out and replace it with a siloed DevOps team doesn’t help that at all. DevOps isn’t about particular toolsets and neither is it about implementing a black ops team to go around operational turf protection. Rather DevOps is a humanistic movement, one which should be almost completely focused on communication, on bridge building and on the identification of common interest. My frustration was lessened somewhat after reading a post by Jez Humble on the subject. Somewhat confrontationally entitled “There’s No Such Thing as a “DevOps Team”, the core thesis of the post was that

the Devops movement addresses the dysfunction that results from organizations composed of functional silos. Thus, creating another functional silo that sits between dev and ops is clearly a poor (and ironic) way to try and solve these problems. Devops proposes instead strategies to create better collaboration between functional silos, or doing away with the functional silos altogether and creating cross-functional teams (or some combination of these approaches).

In this modern world, where the delivery of agile solutions on an ongoing basis is a non-negotiable requirement for success, organizations need to set themselves an objective to cross functionalize as much within their organizations as possible. This is part of the reason that every step up the stack delivers incremental benefits. Automating infrastructure deployment does much to build bridges between application operations and systems operations teams, and creates common goals where formerly they were hard to identify. Similarly the move from infrastructure up the stack to PaaS all of a sudden means that development and operational tasks are developed and achieved with commonality.

Individual functional silos increase the occurrence of problems. Why? To revert to the previously mentioned post:

…functional silos often get created in reaction to a problem (which they inevitably exacerbate). At the beginning of an interview with Elisabeth Hendrickson I posted recently, she discusses working at a product company which was suffering a series of quality problems. As a result, they hired a VP of QA who set up a QA division. The net result of this, counterintuitively, was to increase the number of bugs. One of the major causes of this was that developers felt that they were no longer responsible for quality, and instead focussed on getting their features into “test” as quickly as they could. Thus they paid less attention to making sure the system was of high quality in the first place, which in turn put more stress on the testers. This created a death spiral of increasingly poor quality, which led to increasing stress on the testers, and so on

Functional silos (and a standalone DevOps team is a great example of one) decouple actions from responsibility. Functional silos allow people to ignore, or at least feel disconnected from, the consequences of their actions. DevOps is a cultural change that encourages, rewards and exposes people taking responsibility for what they do, and what is expected from them. As Werner Vogels from Amazon Web Services says, “you build it, you run it”.

So a “DevOps team” is a risky and ultimately doomed strategy. Sure there are some technical roles, specifically related to the enablement of DevOps as an approach and these roles and tools need to be filled and built. Self service platforms, collaboration and communication systems, tool chains for testing, deployment and operations are all necessary. Sure someone needs to deliver on that stuff. But those are specific technical deliverables and not DevOps. DevOps is about people, communication and collaboration. Organizations ignore that at their peril.

I’m really looking forward to the event and, due to some requests from the community, we’ve been able to free up a few more places – if you’re in Vegas next week for either Interop or EMCWorld, and you’d like to come along – fill out the application form here.

I’m looking forward to meeting up and discussing this stuff with fiends old and new – the agenda (as it stands today) is copied below to whet your appetite.

0935-0945 Housekeeping: MC James Urquhart

Track 1: The Data Center

0945-1015 Keynote: Data center design, efficiency and sustainability – Mark Thiele

The data center as an extension of the ability to deliver IT services is also part of the greater “system” that is your IT. All your measures for operating model performance, for delivering manageable capacity and sustainable cost effective operations will be determined to a large extent by the quality and capability of your data center strategy. The pressure of rapidly changing business models combined with increased demand for IT solutions will have a much more dramatic effect on your data center decisions in 2020, than they do even today.

1015-1045 Keynote: Infrastructure for data-centric world  – Val Bercovici

In this talk, Val will review the four data-centric dimensions of disruption to the infrastructure industry.  New trends covered will include software-defined data management, virtual storage controllers, storage capacity futures, parallel workloads, storage-class memories and next-gen interconnects.  Val will introduce the relevance of these important trends as well as their inter-relationships.  Some of these notions will challenge the conventional wisdom of infrastructure possibilities, making Val’s talk often provocative and highly engaging.

Track 2: The Core Components – Compute, Storage and Network

1045-1115 Panel: Operating Systems of the Future – Cloud Stacks

Moderator – Josh Sanderson
Panelists – Jared Wray – Tier3, Andy Knosp – Eucalyptus, Rodrigo Flores – Cisco

Beyond the obvious war of words between the various open and proprietary cloud operating systems, few people are looking far enough into the future to predict how the landscape will look down the line.

1115-1315 Lunch and Switch SuperNAP Tour

1315-1400 Panel: Programmable Infrastructure

Moderator – Larry Carvalho
Panelists – Jonathan Murray – WMG, Pete Johnson – ProfitBricks, George Reese – enStratius, Randy Bias – CloudScaling

Recent years have seen a massive amount of innovation that drives higher densities from chips. Alongside this we’ve seen a move to low-power designs that deliver on the commodity hardware promise.

1400-1430 Keynote: The Future of HyperScale Datacenters – Agility Thru Disaggregation - Jay Kyathsandra, Intel

Today’s hyper scale customers need for increased capacity, flexible  architecture and lower total cost of ownership  is driving the need  to “re-architect” the traditional platforms and move towards aggregation at the “Rack as a system”.  In this session you will hear about the industry trends extending beyond the traditional server by  innovating across  compute, network and storage platforms, optimized for rack scale architecture to meet a broad range of end user needs and OEM implementations.

1430-1445 Open Discussion on Session Topic

1445-1515 Afternoon Tea

Track 3: Controlling our Heterogeneous Future

1515-1545 Panel: Automation and Cloud – DevOps as a Unifier

Moderator – Sam Charrington
Panelists – Duncan Johnston-Watt – Cloudsoft, Jeff Sussna – Ingineering.IT, Tim Prendergast – Adobe IT

As we increasingly move to a services world with large scale automation, organizational needs demand varying levels of control and abstraction. DevOps is slowly gaining traction trying to meet the diverse needs of a large enterprise. In this session, we will discuss how DevOps is helping organizations and the various use cases.

1545-1600 Open Discussion on Session Topic

Track 4: The Economics – Economics of Infrastructure Services

1600-1630 Panel: The Economics and Use Case of Federated Clouds

Moderator – Michael Ducy
Panelists – John Cowan – 6Fusion, James Mitchell – Strategic Blue, Kosten Metreweli – OnApp

With various open source and proprietary infrastructure platforms in the market, we are seeing more and more cloud providers offering infrastructure services. But it goes against conventional wisdom that there will be handful of providers offering services. In this panel, we will try to make some sense in terms of economics of how it is going to play out and also discuss some use cases around the idea.

1630-1700 Keynote: Beyond Cloudonomics – Mega Trends – Joe Weinman

Joe will address the debate on the business value of IT, as well as whether IT—including the cloud—is just “plumbing” or can be strategic.  How can cloud computing—and related technologies such as big data and mobility—contribute to business strategy?  Joe argues that there are four major ways: operational excellence, product and service leadership, customer intimacy, and accelerated innovation.  Joe will also overview his latest research on the economics of network intelligence, including software-defined networks.

1700-1715 Open Discussion on Session Topic

CloudMunch already has Jenkins features, and today is announcing new integrations with both Github and Opscode Chef to package the entire application development and deployment process. In essence with CloudMunch developers have a “one click” path to depoly directly from Github to the cloud using Chef.

The stated benefits that CloudMunch brings to the development process include:

• Get to Market Faster – GitHub-to-Cloud with 1 Click: CloudMunch enables sign-in with a GitHub ID to enable developers to get started with continuous integration, testing and continuous deployment in minutes. With one click, import code repos, automatically detect test settings and set up for a wide range of web apps. Developers simply enter Chef keys and CloudMunch automatically imports environments and nodes.
• Reduce Deployment Complexity: CloudMunch integrates with Chef to import Chef environments and deploy to Chef nodes. Developers can use and extend more than 800 existing Chef cookbooks or use their Chef recipes and build and validate them continuously using Foodcritic. Also, developers can easily create Chef nodes by launching pre-configured CloudMunch Chef images.
• Improve Collaboration and Decision Making: Developers can tap into the full capabilities of the CloudMunch DevOps Dashboard.  Its collaboration features facilitate discussion across development QA and Ops teams, while in-context, real time analytics deliver intelligent build metrics and real time infrastructure monitoring via AWS Cloudwatch and Nagios. It is also plug and play with a broad range of developer-favorite open source and industry standard tools.

MyPOV

Anything that helps efficiencies in the development to deployment process is a logical step. The entire deployment process “should” be automatic and allowing developers to deploy directly from their work location of choice (Github) is a logical step. However there are other steps I the application lifecycle (load testing being one logical example, automated security checks another) and if a developer has to move off platform to enact these steps, only to move back on to deploy, a degree of dissonance opens up that is sub-optimal. In an ideal world, CloudMunch would encapsulate the entire development process all the way through and deliver on that. It has to be said that CloudMunch supports some intermediate step frameworks (JUnit and PHPUnit for testing for example) but this is different from on-platform integrations with third party deep testing providers.

There is also the distinction between application lifecycle visibility and application lifecycle enablement – I’d be interested to see a partnership between a deployment process provider like CloudMunch and a visibility tool like Appsecute (disclosure – I’m an investor) some kind of integration between those two areas could deliver interesting value as developers both gain visibility and collaboration utility across their entire tool belt, but also gain process automation across the stack.

…by providing an SDK and a set of high-level APIs that sits on top of a fabric layer that connects all the different big data components in an optimal way – they’re delivering on the promise of making big data accessible to all.

Just as a reminder, Continuuity delivers a so-called “Big Data application fabric”, that aims to make it fast and easy for any developer to build, deploy, scale and manage Big Data apps. The platform offers a unified experience across the entire application lifecycle from development to DevOps. Continuuity provides pre-packaged building blocks with higher level APIs, Datasets, tooling and documentation that speed the big data application development process. The founders see four specific use cases for the Continuuity platform:

• Getting data in – event ingestion, data queuing and a core transaction engine
• A more traditional PaaS view of application containers- developers write code, package it up and deploy
• Ready established datasets – collections of data, stored down in the infrastructure with a higher level API. Continuuity provides some data aligned with common patterns in apps – time series, counters etc
• Data out- queries, via RESTful APIs and utilizing both preconfigured and self-built user stored procedures

Well after a few months in private beta, Continuuity is rolling out a public beta of both the developer suite and the developer sandbox. Specific parts of the news today are:

• Moving out of Private Beta to Public Beta of Developer Suite
  • Single node version of the Continuuity AppFabric
  • SDK
  • IDE plugins
  • Dataset Patterns to make data modeling and manipulation easy
  • Test harness
• Introducing the Public Beta of the Developer Sandbox
  • Free access to a self-service, cloud based, single tenant, single node version of the Continuuity AppFabric
  • Trial period of 90 days
  • 8 cores, 8GB of memory and 240GB of storage
• Combination of releasing these both to public beta together allows developers to have the full end-to-end Continuuity experience
• VPC and On-Prem are in Private Beta

To really get an idea of the sort of outcomes that Continuuity is delivering customers, I quizzed CEO Todd Papaioannou about an interesting case study from one of the private beta testers. The customer, which runs a social commerce platform, wanted to deliver targeted recommendations and personalization for users visiting their Web properties. They wanted to process events in real time from a variety of different data sources including all activity on their site (user clickstream, seller actions) as well as additional information about their users pulled from the Facebook API. These events, combined with their proprietary customer segmentation algorithms, allowed them to generate personalized per-user product recommendations and then use them at serving time by making queries directly into the Continuuity AppFabric.

Continuuity is providing a very interesting platform, and one which the current preoccupation with big data from all vendors will get a fair amount of attention – I’m looking forward to seeing some case studies from the public beta.

I’ve known about this project for a couple of months now, but was only really made aware of the value of this approach when I attended DevOpsCon in Israel recently. There I took part in many discussions, but the overarching theme was one of building a team culture that encouraged a holistic vie on an application lifecycle – development, deployment and continuing operations.The new Appsecute social stream delivers a tool which really helps to build that culture.

So how does this social stream work? With the integrations that the team have already built, Appsecute has insight into many different tools. It knows, for example, that your system includes some repositories on GitHub, it knows you’re using CircleCi for continuous integration, it knows the production version of your app is running on Cloud Foundry or Heroku and it knows you’re using Zendesk for support. Appsecute argues that it knows about everything that a DevOps practitioner might care about.

The stream is in real time and, of course, shows the events occuring in the services a developer might use but, more importantly, encourages collaboration around those event. An event can be anything, examples that Appsecute include someone pushing code to GitHub, a build failing, tests failing, an application going offline, a support request coming in, a daily summary of the metrics you care about etc. In terms of the collaboration aspects, anyone can write comments against any event – think of it like Facebook for DevOps without the annoying game requests.

But the great thing about the new Appsecute is that it’s a platform – the product has an open API that allows anyone to plug a new system into the social stream. So anyone using a particular service (or indeed, the service vendor themselves) can create a custom connector that can be written and published on Appsecute – allowing anyone else to use the connector. Appsecute will be hosting any connectors that the community writes and open sources.

For now, Appsecute is launching with some connectors they’ve written themselves – GitHub, Zendesk, CircleCi, Cloud Foundry, AppFog and Heroku. But the team plans to keep rolling out addition connectors over time. The product is currently in private beta – anyone wishing to give it a whirl should email Tyler Power, CTO – tyler@appsecute.com

Given the spin out of the developer focused assets, and the fact that VMware is well known for pushing a very proprietary message, one would be forgiven for assuming that this deal is an indication that Puppet is set to lessen its heterogeneous focus and instead focus 100% on increasing the market share for its parent company’s core products. However Puppet Labs EO Luke Kanies quickly came out with a statement to deny this, in an interview with GigaOm, he reaffirmed the company’s commitment to a level playing field saying that:

One thing Puppet is strong at is heterogeneity — we support Red Hat or Windows, physical or virtual servers, public or private cloud

At the same time, Kanies stated that the spin out of the developer tools sees VMware refocus around its traditional core customers, system administrators.

MyPOV

For Puppet Labs this is a great deal – they get significant exposure through VMware’s existing customer base and should be able to convert the attention to revenue relatively quickly – after all the Puppet proposition is both simple and logical, automating provisioning and change management of infrastructure is a great way to reduce costs for IT departments. So it’s certainly a short term win for Puppet, but also for VMware that gets to leverage the credibility that Puppet has within the cloud cognoscenti.

But is it a long term play for the industry? On this point I’m dubious. The rise of DevOps, the burgeoning attention given to PaaS and a focus on moving abstraction as far up the stack as possible would seem to indicate that, over time, provisioning automation will be a little irrelevant – after all, if an enterprise is heavily adopting PaaS which bundles provisioning alongside even higher level operations, how relevant is an automation product by itself?

Of course this isn’t a criticism of Puppet or its product at all – where the industry is today, Puppet provides a valuable and important service, but those amongst us who spend our time trying to peer into a crystal ball to intuit the future of the industry would suggest that as infrastructure becomes more and more commoditized, and customers look for solutions that abstract more and more of the lower level functions away from their responsibility, tools like Puppet and, by extension, companies like VMware with little developer centricity, will more and more become dinosaurs of a recent past.

(Cross-posted @ The Diversity Blog – SaaS, Cloud & Business Strategy)

CloudAve is sponsored by Salesforce.com, Workday and Zoho.

Boundary sells itself as being able to deliver monitoring for dynamic environments of varying types – it does so by creating the real time topology and monitoring packet flow to and from all the different tiers of the application stack. This real time and real world topology map also allows them to deliver metrics around latency between the different tiers of an application, meaning that developers can fine-tune their applications, and application stacks, for the best possible performance.

Some new functionality that Boundary has announced includes:

• Pre-defined integration for both EC2 and Rackspace Cloud where any status or alerts from those Cloud providers can be annotated into the Boundary data
• A universal RSS integration capability enabling any RSS status or alert to be annotated into the Boundary data. This capability is available today to integrate alerts from New Relic, Splunk, Papertrail and others
• A big data store that will enable customers to define longer periods of historical data that they wish to keep
• A full function, free version of the Boundary solution

This last point is kind of interesting, foregoing the usual freemium approach, Boundary is releasing a free option that has complete functional parity with the paid version. The only restriction on the free version is limitations in terms of incoming data quantity and historical data store.

MyPOV

There’s no question that applications, and application stack topologies, are getting more complex as applications are delivered in a modular and scalable way. Given this, services which give a real time insight into performance across different levels of the application stack are compelling propositions. Add to this the fact that Boundary creates a visual and real time representation of the topology and you have a solution that not only provides real value, but also does in a way that appeals to those DevOps practitioners that like a War Games like real time display on a big screen in the corner somewhere.

I’m not a huge fan of freemium, especially for these sorts of deeply integrated services where there is as much hassle (read cost) in setting the service up as there is cost in running it on an ongoing basis – by my mind, the key way for Boundary to on-board customers is to ease the integration pain and really showcase the functionality of the product to anyone passing by.  The Boundary product is visually appealing, fills a valid pain point, and delivers real value for customers – far better to articulate that, than to go for the lowest cost message…..

Part of this lifecycle occurs chronologically across time (at planning, deployment and management stages) and part of it also occurs vertically across different levels of the stack and across different flavors of infrastructure. To this end CLM 3.0 announces a number of heterogeneous and higher stack functionalities including;

• Support for SaaS request & provisioning
• Integrated with Database and App Release Automation
• IBM LPARS platform support
• Integration with VMWare vCloud Director
• Integrated metering for chargeback
• Cleaner, shopping cart UI experience

All in all a broad offering but there is an area not supported by the existing offering and that is application deployment or in today’s modern parlance, DevOps. BMC is seeking to remedy this with the acquisition of Austin, TX based deployment specialist VaraLogix. The DevOps terms is a somewhat ephemeral concept but broadly relates to a coming-together of development and operations roles (formerly a very polarized split) where by developers have a degree of oversight over the operational aspects of their application deployment. Automation is one part of that puzzle (most notably delivered from products like Puppet and Chef) but there is also a lot of management and monitoring functionality that is important to the DevOps role also.

VaraLogix itself creates the “Q” product which automates deployment and configuration management for both .NET and Java applications. It does so with a centralized management console and the creation of a release repository for archiving of all application components.

MyPOV

The enterprise world is slowly (but surely) moving on from a heavily siloed situations where development and infrastructure are two different worlds. At the same time they’re moving to a far more heterogeneous take on technology than previously. These two changes together raise the question about the need for a central management view on IT operations that combines heterogeneous infrasturcture management alongside application development and deployment toolsets.

Given this – the acquisition of an application deployment product by an infrastructure management company would seem to make sense. And conceptually it does. The reality however is that it is very difficult for an existing company to nicely combine an acquired product alongside an existing on. If my concept of how the future will look for enterprises is right, simply offering two vaguely related products from one vendor simply won’t cut it, rather what is needed is a single pane of glass that spans various horizontal infrastructures and vertical components. In the release announcing the acquisition BMC suggests that VaraLogix will be integrated into the new BMC Release Lifecycle Management products – we’ll have to see how that goes.

In the short term the option of application deployment tools might fill the check-boxes of IT decision-makers who want to be able to give a vague nod in the direction of lean development methodologies and DevOps, it’s not really a long-term and future proofed approach. Rather I see the future as being a much deeper infra and application management solution, that’s one of the reasons that I’m involved with Appsecute (see disclosure regarding this) where the vision is to provide tools that give developers the freedom to use whatever infrastructure platforms and tools they want, to have complete control over their development and test environments, and then to still be able to hand over their apps to IT when they go into production. The second part of that is to provide tools that give IT Operations the freedom to control their production environments, and to have complete visibility over the development and test environments as well. In practice this means a broad solution that includes things like access control, monitoring, escalation, deployment and migration that work across a range of infrastructure vendors and development platforms, whatever each developer or business unit happens to be using. Appsecute CEO Mark Cox riffed on this theme in a recent blog post.

If BMC can cleanly tie together CLM tools alongside its RLM products – then there is the possibility that the combined product offering could become very compelling. History would suggest that outcome is probably unlikely.

Another company that has long been chasing this market is CollabNet a vendor of agile focused products and services for development shops. CollabNet (more on them here) delivers these customers a couple of distinct offerings and is today announcing a new approach towards how it delivers and articulates it’s product portfolio – this approach is intended to deliver something I’ve been talking about for awhile now, a holistic end-to-end development environment that includes tools for collaboration and project management at the development stage, a bunch of services to aid the deployment of applications to a multitude of different infrastructure approaches, and tools to aid in the DevOps role of already deployed apps.

Based on what it’s seeing in the marketplace, CollabNet has articulated what it sees as the five key steps that organizations should take on their way to enterprise cloud development as they adopt hybrid cloud IT into the application development and deployment lifecycle. The steps include:

1. embracing the cloud for centralizing access and visibility to tools and processes
2. creating a robust coding community to encourage reuse
3. the codification of development processes by standardizing tools, workflows and processes
4. automation of DevOps practices
5. a hybrid approach by leveraging public and on-premise private cloud resources in a secure, compliant and optimal way

While in principle I agree with the five steps, I’m less convinced that the hybrid public/on-premise approach is optimal for all situations. While it may suit many of CollabNet’s customers, I’m always worried about advice like this which recommends the “one true way”. Anyway – outside of that little critique, I like the message that CollabNet is articulating.

As a whole, the suite is trying to deliver on the promise of Application Lifecycle Management or ALM, a concept long touted but somewhat short on real proof points – ALM is this end to end management of an application – from inception, through creation, onto deployment and ongoing management.

In terms of product offerings to deliver on this promise – CollabNet is announcing;

1. CloudForge: CollabNet’s development PaaS which is built from the previously acquired Codesion public cloud hosting platform. CloudForge helps developers and IT managers instantly develop and deploy software using a hybrid mix of tools, application frameworks and deployment clouds – all with enterprise security and compliance in place
2. The creation of a CloudForge App Marketplace to mimic the add-on approach of other vendors like Heroku
3. ALM and SCM hybrid cloud services: Initially, CloudForge cloud services will be available to customers with on-premise deployments of Subversion Edge and TeamForge. A new Subversion Edge CloudBackup service now provides Subversion data archiving, redundancy and migration capabilities for any on-premise user of Subversion Edge or TeamForge, without leaving their own desktop environment
4. CollabNet says that later in the year, additional hybrid cloud services will be available in TeamForge and Subversion Edge, such as elastic server provisioning for build, test and deployment.

There are a lot of vendors offering new-age development tool sets – CollabNet joins a bunch of others who are also increasingly competing with both PaaS and IaaS vendors as they move more into the deployment and management parts of development. That doesn’t negate the value of what CollabNet is doing, rather it realistically shows they’re entering an ever-more competitive space. It will be interesting to see how they fare with their new approach.

An announcement today from FeedHenry speaks to this fact. FeedHenry provides cloud-based Mobile Application Platform solutions that ease the development, deployment, integration and management of secure mobile apps for business. Their mobile platform-as-a-service allows apps to be developed in HTML5, JavaScript, and CSS and deployed across all major mobile devices from a single code base. The node.js backend service offers a complete range of APIs designed to simplify and secure the connectivity of mobile apps to backend and third party systems which is where this announcement comes in -  they have just announced the creation of a mobile application platform build on top of the Cloud Foundry PaaS. In articulating the need for a solution like this, FeedHenry speaks to my contention about the different development roles saying that’;

developers are building mobile frontend applications separately, and integrating with backend services manually. To leverage the benefits of PaaS for mobile applications, developers need a way to develop, deploy and scale these cross-platform applications on the PaaS in a seamless manner.

Essentially this integration sees FeedHenry continue focusing of frontend functionality, and abstracting much of the backend responsibility to a PaaS provider and in doing so reduces the need for their customers to spend time worrying about infrastructure-level issues. Jerry Chen from VMware speaks to the coming together of frontend and backend operations that offerings like this bring saying that;

We are seeing a shift to mobile-first development within enterprises, which requires their PaaS solution to provide an integrated mobile development and deployment platform. FeedHenry’s cross-platform mobile development solution can make it easier and faster for Cloud Foundry developers to build and deploy these applications.

Of course simply integrating frontend and backend development platforms doesn’t answer all the issues around bringing together these two worlds. But with tight integration, at least the technical barriers to a more cohesive development framework will be overcome. FeedHenry looks to be a good step in this direction.