Building and running an agile delivery organisation – Doing Digital in Bristol pt. 2

It’s taken me a long time to get this post into shape and published. We’ve been living this journey for the past year and have a pretty clear understanding of where we’ve been, what we’ve done along the way and where we are going next. But articulating that clearly for an outside audience is tricky – I don’t want to present a glossy picture to you, that’s not helpful. If I was reading this, I think I’d want to hear what’s gone well and what mistakes we made so that you have a chance to learn and do things differently.

Although I’m responsible for the words used to tell this story, the actual work I’m describing was very much a team effort. My colleague Tara runs the Programme Management Office, and she and I have jointly led this transition to a new approach. It’s her team who are the heart and soul of the service redesign projects. Much of what I’ve written here is based on the learning and developing practices that have emerged in our joint design and delivery teams.

Where we’ve been…
Like all councils, we’ve got years of experience running traditional projects, based around PRINCE2 and a waterfall style solution development lifecycle. We felt that we were fairly good at delivering these projects, and had introduced a number of professional disciplines alongside Project Management – Business Analysis, Enterprise and Solution Architecture.

But the truth is that far too many of our projects took a very long time to come to completion. In fact too many of them took a long time to progress through an initial feasibility and design stage before turning out to be unable to make a business case – we weren’t quick enough at identifying and delivering value for the organisation, or for the public.

Our process involved several sequential stages, with people operating in professional silos, and throwing outputs over the fence to the next silo. We made a few changes along the way to improve collaboration and flow of work, reducing handoffs, but it was all tinkering around the edges. As a result of the combined financial challenge and rising demand for modernised services we ended up with a complex portfolio of 14 programmes, each with many projects, all operating to their own objectives and often cutting across each other, or distracting attention from important common themes.

Over the past few years it’s become increasingly obvious that this approach can’t deliver the results we need – whether it’s because we aren’t close enough to user needs, or because we take too long to deliver working solutions that have value to the organisation, or that the solutions themselves are too heavyweight and based on legacy views of what local government is about

We began to investigate ways of breaking out of the traditional approach in 2013. We had defined a new council wide programme of work focused on customers and process improvement, and within this had recognised that our ambitions to deliver a major shift to online channels meant that we needed to invest in new infrastructure and a new way of delivering services. This is when we began to look seriously at the emerging agile digital methods that GDS were promoting and decided to commit to them for our new online services.

And then came a decisive break with our past, when Nicola Yates joined us as City Director a year ago and led us in radically revising our approach to change, creating a single whole council programme, bringing every aspect of what needs to be improved or transformed in this organisation into one governance and delivery structure.

This gave us a number of major opportunities. We brought all of the work on delivering digital services and the associated process and technology change into one place, with my leadership for architecture and design and Tara’s for delivery. We had an injection of radical thinking courtesy of Tom Steinberg, who came to our board and challenged us to be bold and drive through a user needs focused vision in the spirit of GDS – and we got the explicit support of our City Director to do this.

Nicola also re shaped our approach to justifying investment and agreeing common approaches. Her view was that there would be only one way of doing things in Bristol and that the business case for the whole programme was the key, not individual projects – although of course each and every project has to contribute towards the transformation of the council and the savings we need to make. But taking a view across the whole programme, and recognising that some costs supported benefits in multiple projects meant we could build the infrastructure we needed to deliver digital services.

Delivering our digital services infrastructure
At the same time that we began to drive towards truly digital services, and to build the new infrastructure, we had to create a new delivery team structure to do this, and needed new skills and capabilities. We didn’t have an inhouse development team, so we used G-Cloud to source a digital agency who were to bring agile development and UX skills to the task. We put a project manager in place who had some experience of agile and began to build up roles like product owner.

It wasn’t long before we hit the first of our challenges – we found product ownership and management difficult. Our ability to write good user stories wasn’t good enough, and our suppliers weren’t really strong enough in agile methods or confident enough to be honest about that with us. We found ourselves coming out of sprints with very little done and more stories written than done – almost a negative velocity!

Around this time we recognised the need for an agile coach but didn’t make the move to bring them on board for some months – just because we were too bogged down in the day to day – but in fact once we made the effort to find an appropriate coach it made all the difference – that and the fact we were lucky enough to find Sarah Prag, ex GOV.UK product manager!

So alongside product management, the other key activities and skills we needed in this phase of our evolution included:
a. Solution and technical architecture
b. Commercial management of suppliers within the agile process
c. Testing

As we built our infrastructure and began to design and deliver digital services on it in two week sprints we found a new challenge. Transactional digital services aren’t like building a web site to provide information advice and guidance. They need careful integration and testing before rolling out to the public. Although adopting user stories and agile scrum based techniques can help in some ways they also introduce more issues.

You need to blend user story driven front end development with the often complicated design and build of integrations, procurement of new services or upgrades to line of business systems. When the refinement of user stories during sprint planning uncovers new issues or an improved understanding of what back end infrastructure is needed, you have to redirect staff and suppliers who may be more used to delivering to a fixed scope of work. You need the right contractual basis and the right culture, attitudes and behaviours in your team for this to work well.

You also need to develop the operating procedures with your IT service – whether internal or outsourced. Agile development and delivery won’t work if it bumps up against traditional multi-week change control processes. You need to be able to support multiple environments, continuous build and integration, and an agile testing cycle. Clearly there is still rigorous control over transition to live, but if you want to run public betas they won’t be of the same nature as a traditional live service.

We had lots of adventures in this stage of our journey – wrong turns, confused wandering in the fog, arguments about what was the right direction to head in. It took strong leadership to get the team focused and facing in one direction – both at the strategic level but also from the delivery team manager role.

Part of the problem – with hindsight – is that we moved to an agile approach whilst still building the infrastructure, which would have been better as a standard waterfall project. Once we had the services and technologies in place we could have dived into agile delivery of services with fewer complications.

As I noted earlier, we also didn’t get an agile coach in early enough. Once Sarah was working with us she helped us to focus on the root cause of the challenges we were experiencing. We did this through standard sprint retrospectives. The fundamental issue we identified was that we had moved to delivery too quickly, without a deep and comprehensive enough discovery phase. So we worked with Sarah to define a detailed guide to discovery and she coached the next couple of projects through that stage.

Evolving into Service Design
So the next stage of our journey has been travelled over the past 6 months. The infrastructure is pretty much in place, so our projects are focused on user needs driven, outside-in, service design, enabled by these new digital opportunities. Again we needed to evolve our skills and capabilities, and the shape of the teams that deliver redesigned services

Working with Sarah Prag we’ve really unpacked what Discovery means, identifying three sub-stages – learning, shaping and planning. Within these stages you gather a number of inputs, do a number of things to them, and produce some outputs.

(Sarah has asked me to point out that this visualisation of the design process has been stolen from the Design Council’s famous “double diamond” – her cheeky addition of the ongoing “run and improve” phase is the key difference)

SP-discovery-delivery diamonds

If you look at it with your eyes half closed, the list of activities looks very similar to a typical PRINCE2 project in startup and initiation stages. But there are crucial differences.

For a start there’s the intensive focus on customer experience, rather than internal process. We are absolutely starting from user needs. Of course we do then blend in an understanding of the business function and organisation – this is all framed within the need to reduce our annual operating costs by £85m over the next couple of years so clearly we have to know where the waste and failure is in our operational procedures. But the important thing to take away is that you first work through how the service could be reimagined to meet user needs through digitally enabled means, and then turn to the process. We’ve found that entire chunks of current activity are simply swept away – they only exist because we created a complex set of rules around the limitations of our capabilities and policies created in a pre-Internet world.

Another important difference is that you co-locate a small team of people in one space, and support them to think visually and collaboratively, not write reams of documentation. We’ve loosened the expectations of documentation and allowed our teams to discover what is absolutely needed, and are now explicitly redefining what makes up the standard set of products.

The main roles in these redesign teams are the Project Manager, Business Analyst, CX expert, Solution Architect – and of course the subject matter expert from the service.

And that brings us up to date. We are about to embark on the next stage of evolution in this story. We want to draw service managers into this way of delivering continuous agile digital service redesign. It’s very early days, but we expect to blend the specialist teams we’ve created to date with operational staff and managers to increase capacity and enable a true continuous improvement approach.

We are working on an approach to scaling agile delivery now and will share more when we can.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s