Lots of Small Steps instead of One Giant Leap

My fellow consultants here at CapTech have shared quite a bit of knowledge on agile methodology lately, showcasing its growing popularity and benefits. But, as a data analyst, I found myself a bit hesitant to enthusiastically endorse it for data-oriented projects. Until, that is, I was assigned to an agile project. Needless to say, I learned quite a bit, and am now much more inclined to suggest an agile rather than waterfall approach. From deep in the weeds, here are a few lessons and suggestions.

Have a well-defined epic. It is critical for the entire team to understand the business reasons behind the project. Among the considerations:

  • Why is this project happening? What would happen if the project was never enacted?
  • What problem will it solve?
  • Is there a set of business issues or concerns that will be addressed by this project?
  • Who is the Sponsor?
  • Who is involved?

All of these help analysts and developers stay properly and consistently grounded in the objective, direction, and task priority of the project.

Make sure everyone understands the resulting stories. Often, we get caught up in the "how" rather than the "why". In Agile, we don't always know the "how" until after a sprint begins, but we better know the "why" in order to deliver that which we're being asked. Lack of overall clarity, poorly worded stories, and lack of overall vision can quickly lead to deliverables that miss the mark. Understanding the epic ensures that the stories are correct. Well-written stories ensure that the deliverable provides the desired business value.

Many Developers and Modelers will balk at not having a 100% complete data model before moving forward with development. That's a reasonable concern, but setting expectations up front can help smooth those over. There is no expectation of data model perfection after the first few sprints, and there will be adjustments as the project moves along - just like in the waterfall approach. The difference is that in agile, end-of-sprint deliverables may be placed into production; acknowledging that the deliverable is not perfect. It doesn't mean we built the wrong thing, or that it's broken. It means we are providing the highest priority and highest business value products first.

There will be some "data model evolution," so expect pushback on "change." Developers look for stability in a data model. Re-work is considered a bad thing and a waste of time, especially when "it could have been done right the first time." But that's not how agile works. Change is constant as priorities are identified and shifted around. Anticipate that developers will express their opinions, and be prepared with answers and assurances. Realize that re-work will happen. Plan for it.

Spend working time together as a team each day outside of the stand ups. Spending time in a common area (even if it's an open speakerphone line and group chat window) helps build team unity and forges better working relationships. Having that open line of communication helps resolve issues sooner, build trust, helps share knowledge, and strengthens working relationships - all leading to a more productive team.

Demonstrate Progress and Business Value Delivery Part 1. With proper prioritization, there is some business value produced each Sprint, even if it's pre-work for the next one. When that value is called out, the entire team, as well as those only peripherally associated with it, can recognize and acknowledge progress and business value. In addition, those benefits may well be delivered far sooner than in a waterfall project.

Demonstrate Progress and Business Value Delivery Part 2. As with all projects that require frequent and constant business user feedback and interaction, it's easy for anyone to become distracted or to become less involved on a day-to-day basis. But with agile, constant feedback is important. Using a consistent stream of increased business value is your primary tool to pull them back in as needed.

Note that the tasks involved in agile are the same as waterfall - no difference - only the structure around them changes. You will still perform the requirements gathering, data profiling, ETL, report construction, etc., just in smaller chunks of business value.

About the Author

David Elliott

David Elliott has been helping CapTech clients for over eight years implement data solutions of all types. He is the Data Analysis Lead for CapTech's Richmond, VA office and is passionate about data and its impact on companies as well as individuals.