Software suppliers as a service

We need to shape our organizations wisely and can't have
too many crazy cooks in the kitchen.  We need to create
the role of the master chef for our enterprises.

I've been brought into a few situations recently to help companies where the software they bought was not performing as they had expected it to.  They thought it was going to be great based on what the vendor had told them but when it came down to business and reality, it wasn't doing the job.

There's a problem in industry today and it's probably always been there.  I think our organizations went a little bit crazy in implementing tools and technologies but never thought about the big picture, the architecture, processes, or operations that would support the organization in the long term.

Our organizations have to become more lean and agile but they also have to partner very wisely.  Rather than implementing tool after tool, they have to question the process and way by which they are doing this.  Are they doing it in a controlled way?  Are they doing it in a sustainable and continuously improving way?

There are a lot of tools and vendors out there and they all might be pretty good but organizations need a single service that helps them contain and control technology acquisition costs and partners.  Let's not run amuck and buy a bunch of tools without control or process.  Thanks.

Inversion of Control, Dependency Injection. Awesome!

This stuff seems so cool and important for being able to test your code.  I just watched this very excellent video from James Kovaks on IoC and DI in C#.  He makes it very simple to use and understand the concepts of why and how to do this.  Great job on this.

Workflow foundation: new capture. SSIS: existing capture

Per my previous post on the two faces of business intelligence, I'd like to further my point by describing two "core" technologies from Microsoft: Windows Workflow Foundation (WF) and SharePoint Integration Services (SSIS).  These two technologies in my opinion are "big" in the sense that that "orchestrate workflow".  Let me explain.

Windows Workflow Foundation is a technology that allows a developer to "map out" a business process and wire in the user scenarios to capture--and then store--the data.  The important part here is that we are going from nothing (the user's mind) to something (the database).

SQL Server Integration Services (SSIS) is a technology that allows a developer to "map out" an in-memory data transformation process.  The important part here is that we are going from the database *back* to the user's mind (typically in the form of reports or other data visualizations).

I think the interesting thing to note here is that these two technologies are *so* similar.  They depend on tasks, create flows, etc., but are not really integrated in any real way.  Is one a sub-set of the other?  How do we think about these two different--yet very similar--technologies.  Do they fit within the same developer toolkit or what?

As a solution architect you have to be aware of these things.

Interesting related reading and reference:
http://www.developer.com/db/article.php/3780676/Building-a-Windows-Workflow-SQL-Server-Integration-Services-Activity.htm

The Release Manager is the Product Owner

What's a release manager? A marketer. A public relations fellow. Someone that cares both about the customer AND the product development team but in no specific order. This person is the best possible person you could have on your team or company. They can turn a bad project into a good one and a good one into a bad one. If you have someone acting in a release manager capacity that is either A) all about the requirements or B) all about giving the customer everything they want regardless of cost, you're in trouble.

Train people in this role how to help you out, provide air-cover, and also how to serve the customer. If you are the guy/gal responsible for delivering the product find someone who complements you very well and understands how to communicate, deflect, and the like. I'm not yet sure if the Release Manager needs to be technical in a software project but I believe the answer is definitely yes, since if they aren't they can't deflect and backlog as necessary, that would end up falling to you.

No matter how paranoid you are, you’re never paranoid enough.

Afraid someone is watching you? They probably are.

In a computer system, anything that did once exist (all events and data) could be logged into a database and analyzed either now or later. This data could be used against you but it could also be used for good. How so? In making our software better! But what is the cost?

The more parts of our software that get "instrumented" (hooked up to log actions into a database), the more at risk we are for losing our privacy. And A LOT of the parts of our software are getting instrumented and hooked up to online databases. You could build your house into Fort Knox, but you need a data Fort Knox as well and things like "The Cloud" conflict and compete with that. Many companies today instrument their software in order to learn how people use their software. They use this data to make the software better; but there is a very big cost to this! This same data could be stolen or subpoenaed and used against you. Not exactly what you want.

In short, be very, very careful and very, very paranoid of what you're doing on your computer and online … they could be (and probably are) watching you!!! ; )

The Application Layer is All That Matters

I think that this layer is the only layer that matters.  It is the business process, it is the interaction between the user and the computer.  It is workflow and the movement of data.  It is all about the purpose for which the computer exists in the first place.  It is instrumented, it is intelligent.

List of Data Visualization Tools

Just got "Now You See It" by Stephen Few.  Here's a  list of the tools he's using to generate good-looking visualizations of data in the book:


Panopticon Explorer


Time Searcher 2

Tableau Software

Inxight's Table Lens

Spotfire

Vizster

JMP Profiler

Trixie Tracker

Google Motion Charts

Managing Schedule Risk in an Agile Scrum Process

Managing time and the team in an agile scrum project is difficult but it's gotta get done. In all projects, we have to estimate what we're going to do and try to get it done. If we aren't getting it done, we have to either work more and meet our deadline or communicate to our sponsors how things are going to be different at the deadline or move the deadline out. It's all about delivery and customer service in the end. At any rate, typical software projects have two distinct parts, each requiring their own management techniques:

1. New feature development



2. Bug fix, customer service and support

Since we have to manage both of these types of work, and they behave very differently, we need to develop a language around getting things done to support effective communications within the delivery team and to our customer community. The choice we use at my company is Agile/Scrum, where we plan and deliver our work in 2-week increments called sprints. By doing this we're able to execute and significantly re-tweak our process every two weeks. We are given a quote-unquote new lease on life every two weeks; it should be a refreshing way to work and partner.

In our methodology, new feature development is handled by the Agile/Scrum methodology burndown method to track work. The burndown method is the typical agile/scrum way of tracking work: we create a definition of done in terms of scope and features, then estimate how many hours we think it will take to do it. Between the ScrumMaster and the team, we track how the hours are burning (up or down since maybe your last estimate wasn't great and there was unseen work) on a recurring basis and if we think our velocity is what it needs to be. If the velocity starts to slow down, the team addresses how to remove impediments and get the velocity up to such a degree that the project is scheduled to finish on time. If things don't go well, we figure out how to mitigate the risk with the customer.

Burnup is a different but important concept to understand and use to manage our work. Our example of using burnup here is a 50-hour max promise to the customer to deliver customer service. We don't have to provide all 50 hours in our example but we do want to know how much of this time we're spending because it's a distraction to the other part of the project since we can share resources and sharing resources is efficient. This method requires tracking ACTUAL hours spent, which is different from the burndown method. Note that this is a key difference from burndown, where we really only care about how many hours are left. We can see how many hours are left on a burndown type of project but what's potentially left on it is certain, whereas what's left in burndown is only estimated.

These two methods of burnup and burndown are two very different ways to think about work: the former is a picture of what's necessarily remaining and the latter is a guess of what's left. The latter is best mitigated by good planning and knowledge of what tasks and how much time is required to get to "done". Together, we use these two techniques to manage risk and communicate effectively.

Conclusion: We Can Now Manage Projects That Burn Time Up and Burn Time Down

By understanding how these types of work are different and how we can manage each, we're able to develop and speak a language about risk and progress making our velocity increase and our projects more fun.

Making Companies is like Making Movies

There's a lot to be learned from Hollywood for the production of software. Making software isn't (or shouldn't be) all that different from making films. Since the early 1900's, Hollywood has been pumping out film after film. Technologies have of course changed over the years but the end product of a motion picture remains. Software is similar. Hollywood's "simple" process of pre-production, production, and post-production is a powerful tool. And its simple roles of Scriptwriter, Producer, Director, Actors, and Post-Product workers are powerful if applied to software projects.

PRE-PRODUCTION ROLES:
Scriptwriters
In film, these are the story tellers.
In software, these are also the storytellers. Their titles, however, are Business Analyst, Product Manager, Requirements Engineer, or Process Engineer.

Producers
In film, these are the guys that get the money and form the team.
In software, these are the Project and Program Managers (or above) who get the money (from investors or internal sponsors) and form the team.

PRODUCTION ROLES:

Director
In film, the director is responsible for the creative work and directing the actors.
In software, the IT Project or Program Manager does the same thing. The actors in software are the designers and developers who are creating the end product for the customer.

PHASES:
Pre-Production
In film, this is creating and refining the script, getting the actors, having a plan, and budget.
In software, this is creating and defining the software, getting the developers, having a plan and budget.

Production
In film, this is "shooting" the film and having the actors act according to the (sometimes changing) plan.
In software, this is "coding" the software and having the developers code according to the (sometimes changing) plan.

Post-Production
In film, this is the work involved in creating the final product of sufficient quality.
In software, this is QA work and is generally iterative where things are returned to developers to fix.

Iteration
In film, iteration is common. Many "takes" happen.
In software, iteration is common. Much "rework" happens.

So in total, I hope you can see that a patter and relationship emerges between film and software. They're the same. They're both productive processes generating a product for an audience. Applying more of our knowledge of the film industry and how it works to software would improve development lifecycles, decrease budgets, and improve user-satisfaction.