A journey in search of Agility

Ateliê de Software’s journey in search of agility, thousands of sticky notes later…

This is the story of Ateliê de Software and its long journey in search of Agility. We have heard countless times that “Agility is dead” or that “Agility doesn’t work”, so we decided to write this account to deepen the discussion on the subject. Our purpose is to show that “agility” is simply a characteristic that an organization or team develops when they are able to continuously learn and adapt to create value for the business and for customers.

Once upon a time…

Around 2006, my partners and I worked at a “Software Factory” that ran software development projects for the government and large companies. This factory held several certifications, such as ISO9001, ISO27001 and CMMi 3, as well as an extremely structured software development process, with well-defined roles, artifacts and procedures.

Every project started with the “Business Analysis” team, which described the customer’s goals and also its main business rules. Then came a “Requirements Analysis” phase, in which the functional and non-functional requirements were gathered, described and documented by another team that specialized only in that activity.

Next, the “Systems Analysis” team specified the use cases and their respective low-fidelity prototypes. In parallel, the design team began developing the interface, working out the visual concept and defining the user interaction patterns.

By the time a project reached this point, about six months had passed since it started. Only then did the team of programmers begin implementing the use cases, while the testing team began specifying the test cases. At the end of each use case’s implementation, the developers sent the finished work to the testing team for verification and validation.

At the end of the project, the software was delivered to the customer to be deployed and used by its users. However, besides having a few defects, the software turned out to be practically useless, because the context had changed and the users’ needs were already different from those identified at the beginning of the project.

After almost two years of work, an investment of thousands of reais and more than 15 people involved, we couldn’t deliver software that truly added value for the customer and its users. The factory had a software development process that was efficient and profitable only for its own business model, but that failed to be efficient and effective at creating value for its customers’ businesses.

On the edge of the abyss

In 2007, the software factory faced an unprecedented crisis. The death of the founding partner, the loss of important customers and the departure of experienced professionals brought the company very close to bankruptcy.

In response, we launched a major sales effort to win new customers and keep the factory running. In that context, a foundation, in partnership with a large Brazilian state-owned company, presented us with the challenge of building software to support a renowned national award for quality in management.

This was the project the factory needed at that moment. However, there was one crucial detail: the software had to be finished in just six months, because a date had already been set for companies from all over Brazil to use it to take part in the award.

Knowing that the software factory’s predefined development process was completely unworkable for this scenario, and standing on the edge of the abyss, we decided to take a risk and adopt a different approach to work and management.

At the time, some people at the software factory were already studying agility and agile methods, and this looked like the ideal opportunity to try an iterative and incremental approach to software development.

So we decided to use Scrum to manage the project. We formed a multidisciplinary team, with analysts, programmers, designers and testers working together. We held weekly meetings with the customer, who was always present, validating every feature we implemented.

That way, the project moved forward quickly, driven by continuous feedback and learning, which allowed us to meet the deadline and deliver the software on the date the customer wanted. Companies from all over Brazil downloaded our solution and used it without major problems. It was the first truly successful project we witnessed at that software factory.

No one is a prophet in their own land

After the success of that project and the software we built, the team of analysts, programmers, testers and designers was excited to keep working that way, which was much lighter and brought better results for the customer and for the factory.

Then, at a meeting of the Software Engineering Process Group, the factory’s director told us we would no longer run projects that way, using that kind of method. He stressed that we had to strictly follow all the predefined certifications and processes, given the large investment made in those certifications and practices.

Despite all our arguments and the indisputable evidence of the efficiency and effectiveness of the agile approach to software development, despite the project’s success and the customer’s satisfaction, the software factory chose not to work with agility anymore. So we decided to leave and start a completely different journey.

Ateliê de Software

And so, in 2008, my partners and I founded Ateliê de Software, a company designed to run custom software projects using agile methods.

The name was chosen on purpose, because it contrasted with the idea of a factory-like organization and reflected our view of how software development should be approached: a combination of art and engineering; technique and creativity; experimentation and learning.

Despite our excitement about this new vision, the reality was that we had only one experience with agile methods and needed to learn a lot to reach a level of excellence that would ensure the success and prosperity of Ateliê de Software.

Agile Project Management

Our journey in search of agility began with adopting Scrum to manage projects, an agile method we already knew from our days at the software factory.

So we decided to take the Scrum training offered by Caelum (now Alura), and we had the privilege of being trained by AleXandre Magno, who at the time taught the courses for the Scrum Master certification.

In Ateliê de Software’s first projects, we realized that planning on a recurring basis and focusing on delivering working software at the end of each Sprint really did improve the flow of value to the customer. We learned to work as a team, to communicate effectively with customers and to keep our relationships transparent.

In those early days, we also had to learn how to sell agile projects and draw up commercial contracts compatible with this way of working. For that, we turned to the experience of Vinícius Teles (viniciusteles) and used as a reference the contract model he created at his company Improve It.

Despite this promising start, our software deliveries were still low in quality, with many bugs coming back to be fixed in the following Sprints. Although we had managed to sell and manage projects and teams in an agile way, we still faced significant challenges with quality in software engineering.

Agile Software Engineering Practices

So we decided to invest in agile software engineering practices, such as TDD (Test-Driven Development), Pair Programming, Peer Review, Refactoring, Continuous Integration, Continuous Deployment, Continuous Delivery and many other good practices that were emerging or becoming established in the first decade of the 2000s.

For that, in 2010, we had the support of our great friend Klaus Wuestefeld, who ran an immersion in eXtreme Programming practices with us and helped us significantly raise the quality of our deliveries.

On top of that, we bought Kent Beck’s TDD video course, which helped refine our test-writing techniques. We also had the opportunity to take part in Maurício Aniche’s research on TDD, which brought new knowledge to our team and showed how important it is to drive the design and development of software through automated unit tests.

After almost two years using Scrum for project management and XP’s agile practices to deliver quality software, we ended up discovering a new “pain point” in our work process…

Agile Business and Requirements Analysis

Having good project management and delivering quality software revealed the need to also adopt agile practices for Business and Requirements Analysis.

In general, we noticed that the companies asking for custom software development had great difficulty determining which problems the software needed to solve, as well as which business goals the project should achieve.

To help our customers with these definitions, we sought the guidance and knowledge of our friend Luiz C. Parzianello, who at the time was already presenting an agile approach to business and requirements analysis based on the BABOK (now known as the “Agile Extension to the BABOK Guide”).

We also adopted Paulo Caroli’s Lean Inception method (which at the time was still called “Direto ao Ponto”) to design MVP projects and more consistent technical proposals. For that, we had the honor of being trained by our friend Yoris Linhares, from Orgganica, who taught the Lean Inception training to the whole Ateliê de Software team and to representatives of our customers.

To sharpen our skills in this area, we also took the “Business Analyst Training” program run by our friend Paulo Vasconcellos, who shared his knowledge and experience, helped our teams improve their business analysis work and followed the management of our projects for a few months.

That’s how we learned to deal properly with business, software and user requirements, greatly improving the quality of our planning and also the team’s and the customer’s awareness of the value the software was meant to deliver.

Product UX Design

Another aspect that contributed a lot to gaining agility in Ateliê de Software’s projects was the search for agile approaches to product design (now quite widespread).

While we were learning to work with agile software engineering practices, we had our first closer contact with the world of product design through the work of Frederick van Amstel (better known as Usabilidoido) and his DRIFT training. We discovered what user-centered design really was and how to use it in our projects from the very beginning.

We also had the privilege of counting on the knowledge and guidance in UX Design of professor Tiago Silva da Silva, who helped spread the main ideas of usability at Ateliê de Software and also refine the design practices in our projects.

That’s how we learned that Business and Requirements Analysis failed when it didn’t deeply consider the perspective of the users, who were the people who would actually use our solution day to day. So we started doing user research and interviews, creating personas, mapping their journeys through business processes and running usability tests with the help of high-fidelity prototypes.

After almost 5 years of a lot of learning, Ateliê de Software’s projects already relied on Agile Business and Requirements Analysis, Product UX Design, Lean Inception, Scrum and XP’s agile Software Engineering practices. Even so, we still needed to keep improving the way we worked…

Workflow Management

In 2013, to increase the efficiency and effectiveness of our software development projects, we decided to adopt Kanban integrated with Scrum.

Our interest in Kanban came after Alisson Vale’s talk at Agile Brazil 2010, in Porto Alegre, where we understood the importance of managing queues and dealing with bottlenecks and waste in software projects.

Then we took the Kanban Method training with Rodrigo Yoshima, from Aspercom, which gave us new perspectives on flow management and helped improve our delivery cadence.

We learned to deal with UpStream and DownStream, and to use flow metrics and indicators to optimize our work process.

We also got acquainted with the concept of Unified Flow (“Fluxo Unificado”), created by Rafael Cáceres and Celso Martins from Taller, and started applying it in specific contexts at Ateliê de Software.

Kanban helped refine our agility, enabling more efficient management and even more effective software delivery.

Organic Management Style

Throughout this whole journey in search of agility, an organic management style ended up emerging at Ateliê de Software, centered on people and their relationships. Our way of managing is guided by shared and distributed decisions, oriented by rules, constraints and agreements established among everyone involved in the business.

Built on stable, high-performing autonomous teams made up of experienced professionals who use agile methods and practices to produce quality software, our management style is anchored in values such as Autonomy, Collaboration, Self-Organization, Transparency, Excellence and Learning.

We don’t have a formal hierarchical structure, but we value emergent leadership to deal with the challenges we face. We have an organizational culture that promotes an environment of psychological safety and encourages feedback for professional development (we even created the Feedback Canvas, a tool to make the feedback process easier in the context of teamwork).

Ateliê de Software’s management style was shaped over the years with the support of great names such as Ricardo Semler, Niels Pflaeging, Frederic Laloux and Jurgen Appelo. Through the books, courses, advice and consulting of these professionals, we found a way of managing that is compatible with the values and principles of the Agile Manifesto.

As a result of this focus on management, in 2012 Ateliê de Software won the Prêmio MPE Brasil, an award run by SEBRAE. At the time (when we were still using the name Webgoal), we were recognized as the best IT company in the State of São Paulo and also received the innovation award, standing out for our management style (which met all the criteria of excellence and quality in management listed by SEBRAE, but implemented them in a way that was very different from the traditional one).

A journey that isn’t over yet

Today, after 16 years, dozens of projects delivered and many fantastic professionals who lived through this journey in search of agility, we are happy to share this small success and to thank all the customers who bet on this modern way of developing software: Conlicitação, W1 Consultoria, Synergia, Grupo Bandeirantes, Lumiar Educação, Granatum, Access Loans and so many others.

Right now, we are improving our work in Product UX Design (to help conceive our customers’ digital products) and Software & IT Infrastructure Design (to invest a bit more time at the start of projects in defining Software Architecture and IT Infrastructure, given the complexity of the solutions and their need to scale).

We are also learning to use artificial intelligence tools to be more productive and raise the quality of what we do, improving our customers’ return on investment.

Finally, we are already getting ready to serve the government, aware of the limitations the law imposes on agile methods and practices. Perhaps we are witnessing the beginning of a new chapter in this journey, one in which we will truly and finally bring agility to public sector software projects. 😉