Pegasus Experiment
Organizational/Process Design
Design Lead/ Creator
Disclaimer
The following case study is personal and does not necessarily represent IBM’s positions, strategies, or opinions. I have omitted and obfuscated confidential information.
Client
IBM
Timeframe
July 2019- May 2020
Role
Design Lead/ Creator
Type
Design Ops, Organizational Design, Management
Overview
When I joined the Watson Assistant design team in 2019, the team was operating in a legacy structure that was inefficient and counterproductive. As a Lead Designer, I faced this challenge head-on and created a new operational model meant to help the team deliver quality design work while leaning on each other's strengths. We called this new model the Pegasus Experiment.
My role
I created and rolled out this experimental organizational model with the support of upper-level management across the design team.
Challenge
At the beginning of 2019, our design team found themselves in a problematic situation. The designers were working in legacy team structure comprised of squad assignments where each designer worked with a single development squad, and each development squad was organized around a single feature set within the product.
This structure created a few problems for the design team, the first being that the single designer assigned to a squad was responsible for all of that squad's design needs regardless of that designer's core discipline. UX designers were performing visual design tasks, visual designers were performing UX tasks, and all designers were attempting to execute research. While stretching core skills and embracing T-shapes is a good practice, we weren’t applying the skills we had in the right places. We were letting our teammates on the squads down, and the strain on those relationships was showing.
Core problems
Utilization
The squad assignment model was causing lopsided utilization- some designers couldn’t keep up with the demands and needs of their squads on their own, while others were being underutilized. The over-utilized designers struggled to find time to work with their design teammates.
Hyper-specialization
The silo-ing of designers by feature sets also further exasperated this problem as the designers became experts on their focal area of the product, but did not have enough deep expertise in other areas of the product to be able to assist on other assignments without some amount of ramp up time. The silo-ing was causing the designers to lose sight of the holistic product, and encouraging a deep focus on specific features.
Coverage
This also meant that if a designer was out on leave or took time off, there was a hurdle to overcome in terms of onboarding another designer to help take over work in their absence.
Approach
I took a design thinking approach to understand and build solutions around our core problems. I started by working with the team to break down our legacy organization model into an as-is journey map. Each designer was mapped to the squad they supported individually, and a high-level story of a day in the life of a designer emerged. Through this process opportunities and gaps were identified within the existing organizational model.
Half of the design team was given permission to form a new team, called Pegasus. Pegasus’ first task was to analyze the gaps and opportunities identified in the legacy structure as-is journey map and brainstorm experimental solutions. Through this analysis and brainstorming process, the Pegasus team created a vision that described how this new design team should operate to close existing gaps and take advantage of opportunities before us.
Pegasus’ vision statement described a design team which leverages flexible assignment, and alternative process and collaboration methods to reduce the siloing effect of individual assignments. With this experiment, we wanted to explore ways of working better together and supporting the development squads more effectively, while simultaneously encouraging collaboration and embracing each designer’s strengths while coaching them on their weaker areas.
Once our experimental team was formed and had established a vision, we were ready to start creating change. Pegasus had been staffed with four designers. These four designers would support 3 development squads. The squads were selected based on their overlapping and strategically related feature sets.
The designers were selected based on their knowledge of each of these given feature areas, and their unique skill sets. In Pegasus, there was one Visual designer, and three UX designers: one with a research focus, one with an interaction design focus, and one with a UX strategy focus. Together these designers complimented one another’s skillsets and created natural pairings and opportunities for strengthening weaker skillsets.
Goals
As the Pegasus team embarked on the experiment we established a set of goals which were as follows:
A better 3-in-a-box
We believe that collaboration is core to the creation of successful and effective software. We acknowledge and embrace that our counterparts in other disciplines possess skills that can aid us in the design process to create a truly differentiating product. Because of this, our 3rd goal was to create a more robust 3-in-a-box design process that brings development and offering management participation to the forefront and fosters a shared sense of direction and user focus within the squads.
Healthy & supportive team
Great teams can create great things, and central to forming a great team is establishing mutual respect and support among its members. Our 2nd goal centered around creating a space where all members of our squads- regardless of discipline or experience level- can lean into vulnerability, embrace risk-taking, and communicate with one another constructively and openly.
Accessibility
Accessibility in software is of critical importance. As members of the Pegasus squad, we wanted to be advocates for placing accessibility at the forefront of the design and development process and advance all of the human condition regardless of age and ability- this meant adhering to and helping to advance IBM’s usability standards.
Articulate design ROI
Our 4th goal for this model was to define design specific business results that clearly articulate the value of design and its impact on the product.
Increased awareness
Lastly, we wanted to foster a culture of knowledge sharing among the designers on the broader team to align on the purpose and strategy of all the work that is being done throughout the product.
Implementation:
Within a few weeks, the Pegasus designers began making radical changes to processes within the squads to support our goals. A few highlights included:
Establishing formal designer pairing relationships to enable more peer education and support.
Infusing Design Thinking exercises into day to day squad activities to increase efficiency by bringing decision-making squad members together and engaging them in the design process early and often.
Establishing set collaboration hours with squads to aid in efficiency and focus.
Implementing flexible assignment. The flexible assignment allowed designers to move between a subset of related development squads as needed. This flexibility ensured proper coverage and the right skills are matched to a given project
Establishing new lines of communication for collaboration by engaging with squad-mates in new ways and with higher frequency.
Established a brown bag lunch education series to share information skills within the design team more widely.
But it wasn’t just the Pegasus squad that was enacting change. We saw an incredible response from the squad members as well! Some highlights included:
Product managers and developers beginning to invite the designers to help facilitate technical discussions with design thinking activities.
Squads began decreasing the volume of meetings they had in favor of making more time for inter-disciplinary collaboration sessions.
There was an increase in non-designer participation in the design process, which directly correlated to our establishment of collaboration hours and our dedication to bringing others into the design process.
There was also an increase in general awareness of design work among non-design squad members, establishing a deeper understanding of what design does and how we work.
Results
I wanted to quantify and measure the impact we were having on the team and analyze whether we were truly achieving our goals and closing the gaps in the old process. After working in the new formation for 3 months, we created and distributed a design evaluation survey to compare the performance of the experimental Pegasus model to the performance of the legacy model which half the team was still operating within. We had 68 responses to our survey which was comprised of 20 questions meant to evaluate the health and satisfaction of cross squad relationships and the perceived effectiveness of the design. The results were telling.
Squads working with the Pegasus designer model evaluated design 20% higher on average compared to the legacy model.
The success of the Pegasus model has spurred an initiative to create a new repeatable process model that can be applied across teams and measured. This model helps teams ship better products faster and in a less expensive way. These processes are expected to reduce risks, unknowns, and uncertainty, particularly when planning work.
What people are saying…
“Ashley spearheaded an experimental design process within the Watson Assistant team [which] has led to significant quality improvements for interdisciplinary squad members, squad productivity, and individual design experiences. […] The outcome, in turn, reduces risks of product design/development [misalignment], increases velocity, and ultimately results in shipping better products faster! ”
— Design Manager, Watson Assistant