Etsy, an internet market for distinctive, handmade, and classic gadgets, has
seen excessive progress during the last 5 years. Then the pandemic dramatically
modified buyers’ habits, resulting in extra shoppers buying on-line. As a
end result, the Etsy market grew from 45.7 million consumers on the finish of
2019 to 90.1 million consumers (97%) on the finish of 2021 and from 2.5 to five.3
million (112%) sellers in the identical interval.
The expansion massively elevated demand on the technical platform, scaling
visitors nearly 3X in a single day. And Etsy had signifcantly extra clients for
whom it wanted to proceed delivering nice experiences. To maintain up with
that demand, they needed to scale up infrastructure, product supply, and
expertise drastically. Whereas the expansion challenged groups, the enterprise was by no means
bottlenecked. Etsy’s groups have been in a position to ship new and improved
performance, and {the marketplace} continued to supply a wonderful buyer
expertise. This text and the subsequent kind the story of Etsy’s scaling technique.
Etsy’s foundational scaling work had began lengthy earlier than the pandemic. In
2017, Mike Fisher joined as CTO. Josh Silverman had not too long ago joined as Etsy’s
CEO, and was establishing institutional self-discipline to usher in a interval of
progress. Mike has a background in scaling high-growth corporations, and alongside
with Martin Abbott wrote a number of books on the subject, together with The Artwork of Scalability
and Scalability Guidelines.
Etsy relied on bodily {hardware} in two information facilities, presenting a number of
scaling challenges. With their anticipated progress, it was obvious that the
prices would ramp up shortly. It affected product groups’ agility as that they had
to plan far upfront for capability. As well as, the info facilities have been
based mostly in a single state, which represented an availability threat. It was clear
they wanted to maneuver onto the cloud shortly. After an evaluation, Mike and
his staff selected the Google Cloud Platform (GCP) because the cloud companion and
began to plan a program to maneuver their
many programs onto the cloud.
Whereas the cloud migration was taking place, Etsy was rising its enterprise and
its staff. Mike recognized the product supply course of as being one other
potential scaling bottleneck. The autonomy afforded to product groups had
prompted a problem: every staff was delivering in several methods. Becoming a member of a staff
meant studying a brand new set of practices, which was problematic as Etsy was
hiring many new individuals. As well as, that they had observed a number of product
initiatives that didn’t repay as anticipated. These indicators led management
to re-evaluate the effectiveness of their product planning and supply
processes.
Choosing a companion
Etsy had by no means labored with a software program improvement consultancy. To give you the option
to scale product supply and herald some outdoors experience, they began
on the lookout for a vendor. After an intensive search, they chose Thoughtworks,
primarily as a result of they perceived an in depth cultural match and deep alignment
concerning fashionable software program improvement.
Thoughtworks has the same method to Agile as Etsy – being Agile
quite than doing Agile. Etsy’s staff was not trying to undertake a
methodology that blindly follows strict rituals and practices, and wished
a companion that bases its method on agile rules and tradition.
Thoughtworks is thought for technical excellence and management in DevOps,
Steady Integration, and Steady Supply, all of which Etsy depends
on closely. As well as, the 2 corporations shared comparable rules
concerning worker care, variety, open-source, and expertise
sustainability.
The Thoughtworks staff began by embedding into product groups, working with
each purchaser and vendor groups, on varied important initiatives equivalent to cost
mannequin modifications, Search Engine Optimization (search engine optimisation), and a notification platform.
This enabled Thoughtworks to grasp how Etsy labored, created a right away
affect on supply, and improved improvement practices equivalent to automated
testing.
For Thoughtworks, Etsy was fairly completely different from the everyday consumer.
They’re a completely digital firm, and their technical course of and product
practises are very mature. Not like most purchasers who interact Thoughtworks,
Etsy had no massive transformation to undertake, and no ingrained habits
that wanted a serious shift. What they have been in search of was outdoors perspective
and supplementive abilities, to assist them incrementally enhance and sort out
their urgent initiatives. The tradition and rules have been already very
aligned with Thoughtworks’ personal, so the partnership grew to become extra of an
alternate of concepts drawn from completely different experiences.
Scaling Product Supply and Discovery
Thoughtworks and Etsy fashioned a cross-functional Product Supply
Tradition (PDC) staff to investigate the present product supply and discovery
course of. They analyzed your complete software program improvement worth stream,
trying intently on the complaints of managers and staff members. Chosen
occasions have been replayed to achieve a deep understanding of “how” it occurred
quite than who?. The Thoughtworks
staff, as a result of they hadn’t lived by the issues, have been in a position to convey
an goal outsider perspective.
There have been various issues Etsy was doing rather well:
- Cross-functional groups: They structured their groups round “4 desk legs”:
Product, Design, Engineering, and Analytics. All planning and supply practices
occur with collaboration among the many teams. - Incremental supply of worth to customers: Steady supply is the core
observe of their method. A code change sitting in a repository and never
deployed to customers is just not priceless, and successfully prices the corporate cash. Etsy
deploys a whole lot of instances a day, with a 30-minute decide to manufacturing lead
time. - Options are prioritized utilizing information: Product managers and information analysts
collaborate to find out an anticipated worth, measured by KPI modifications. If the
function doesn’t ship sufficient worth versus its value, as evidenced by an A/B
take a look at, it isn’t rolled out, and the code is eliminated.
The staff additionally uncovered alternatives for enchancment:
- Every staff was doing supply in several methods, having disparate processes
and rituals, making onboarding new staff and altering groups tough. Some
groups have been utilizing a course of like Scrum, whereas others didn’t use any formal
course of. - There have been various initiatives that had not created worth as
they anticipated. For instance, Etsy tried a present wrap function that had
decrease than anticipated adoption charges by sellers and consumers. - Previous to 2018, Etsy had been by a transitional interval, with
pivots in product technique and management modifications main some product
managers to expres a want for better possession over their product
selections.
Time to studying
The staff created an enchancment program, based mostly on lean considering and the
concepts of Marty Cagan, who had not too long ago offered at Etsy. The management staff was
studying his e-book Impressed.
I promise you that not less than half the concepts
in your roadmap usually are not going to ship what you hope. (By the way in which, the
actually good groups assume that not less than three quarters of the concepts received’t
carry out like they hope.)
After some preliminary analysis the staff got here up with a metric they referred to as Time
to Studying – the time it took for a product staff to validate an concept with a
buyer and achieve a greater understanding of its worth. They’d a baseline of
50 days that they wished to cut back.
They have been additionally different metrics.
- Enhancing the time to market
- Enhancing the effectiveness of merchandise (e.g. income, buyer
engagement) - Worker happiness
To affect the KPIs, they got here up with various answer hypotheses:
Lightweight prototypes that incorporate extra direct consumer analysis
Etsy has sturdy experimentation infrastructure and analytics capabilities.
They run many concurrent A/B exams on {the marketplace}. The issue the PDC
staff noticed is that the suggestions cycle could be fairly sluggish. For an concept to be
prepared for an A/B take a look at it needed to be close to production-level high quality. It additionally had
to have sufficient information to indicate statistical relevance. A/B exams would run for
months in lesser-used components of {the marketplace}.
To set your expectations, sturdy groups usually
take a look at many product concepts every week—on the order of 10 to twenty or extra per week. I
need to emphasize that these are experiments, usually run utilizing prototypes.
A prototype is just not one thing that’s prepared for prime time and positively not
one thing your organization would attempt to promote and stand behind. However they’re
immensely helpful, as they’re all about studying quick and low-cost.
To cut back the Time to Studying metric, the PDC staff utilized a lean UX
method. First, they began by shortly creating lo-fi prototypes and
instantly displaying them to customers. Second, the staff elevated consumer suggestions
classes to each two weeks and diminished the extent of ceremony. Third, they
created a dual-track system, with a design and analysis staff frequently
working forward of a supply staff. The concept was to effectively achieve confidence
in an concept earlier than it qualifies for an A/B take a look at. By lowering the price of
experimentation, they might improve the range and amount of experiments,
discovering concepts that confirmed extra worth.
Determine 1: Experiment validation course of
Blueprint for product supply
The PDC, along with product staff leaders, got here up with a set of
product supply rules and associated supply observe modifications. The
practices included: smaller extra targeted tales, redesigning story partitions to
higher match the staff course of, utilizing spikes to cut back technical uncertainty, and
higher methods of managing tech debt. They didn’t need to take away the autonomy of
groups. Steady enchancment is a giant a part of Etsy tradition. These practices
are supposed to be handled as smart defaults that may be tailored and improved
for every staff’s particular context.
The PDC staff first examined the observe modifications on pilot product groups to
set up their effectiveness after which iterated. As soon as the observe
demonstrated an enchancment, the staff added to a data base and supplied
mild teaching to introduce the brand new practices to all product groups throughout
Etsy.
Lean Portfolio administration
Some massive initiatives have been nonetheless being carried out, regardless that there was
not clear proof of the initiative creating worth. That is the everyday sunk
value downside. To resolve this, the PDC staff
urged taking a lean method to portfolio administration.
Our runway needs to be an inventory of hypotheses to check, not an inventory of
necessities to construct. After we reward our groups for his or her capacity to ship
necessities, it’s straightforward to quickly bloat our merchandise with pointless
options—resulting in elevated complexity, increased upkeep prices, and
restricted capacity to alter. Options delivered usually are not a measure of success,
enterprise outcomes are. Our runway is a collection of questions that we have to
take a look at to cut back uncertainty and enhance our understanding of progress
alternatives.
As an alternative of making prolonged venture specs and plans, the PDC staff
coached product groups to develop consequence statements for every initiative. With
consequence statements, beginning with a buyer or enterprise consequence permits the
staff to experiment with many alternative concepts to attain the end result by tying
them to measurable KPIs. The much less exact an consequence is, together with its KPIs,
the more durable it’s to specify. When getting ready for attainable initiatives, every
product staff created a two-page transient summarizing the end result, the measurable
affect, and the preliminary funding required.
The manager staff prioritized the end result statements, and every assertion
was seen as a “wager” amongst a portfolio. Impressed by the Lean Worth Tree
idea, the chief staff would observe a daily cadence to find out if a
wager was paying off, giving them the chance to pivot early.
Determine 2: An instance initiative assertion
Stronger product & engineering collaboration
One other enchancment was to incorporate engineers early within the product ideation
course of and to have them be part of consumer suggestions classes. The staff examined
engineers becoming a member of the speculation technology and validation classes and
working intently with designers throughout design sprints. The outcomes of the take a look at
confirmed a number of advantages. Engineers improved their understanding of the
buyer, constructing empathy for his or her wants. They might present on-the-spot
suggestions on the feasibility of an method and shortly create prototypes to
reveal concepts to customers. Cross-functional engagement allowed engineers to
contribute various nice concepts that made it to the product.
After the preliminary PDC initiative Etsy continued to strengthen the
collaboration between teams. At each degree of the group, product and
engineering are companions, which suggests they’re in a position to have wholesome trade-off
discussions. The platform group expanded using product managers for
infrastructure and developer tooling groups – encouraging groups to consider
technical capabilities as merchandise and end-users being Etsy engineers. The
technical product managers do analysis and prioritization actions, simply as
they might for a buyer product.
What was the affect of product supply tradition?
Testing with pilot groups indicated that the Time to Studying went from 50
days to five days. In the course of the pilot, one product supervisor mentioned, “What I really like about
the [direct user research sessions] pilot is that it lets us take a look at our concepts
quicker, it was very nice to not spend as a lot time diving deep into
one thing that we didn’t even know we have been going to make use of.” An initiative on
stock high quality that was predicted to ship a seven million greenback income
improve as an alternative delivered eleven million {dollars} in incremental worth utilizing
the PDC urged enhancements. Following this pilot success, the staff coached
your complete product improvement group on the product supply method.
In line with Mike Fisher, “It is extremely exhausting to isolate the advantages of a
course of change on a company of a number of hundred engineers since we don’t
run A/B experiments of this kind. Nevertheless, we did take a look at excessive degree
productiveness metrics after having PDC carried out for over 2 years, and all of
our metrics have been very constructive. We had improved our velocity by doing this
work, and we have been in a position to produce finally extra advantages for the consumers and
sellers – extra options, higher search, higher processes; the entire
market grew to become higher for consumers and sellers and that was the purpose.”
Etsy, an internet market for distinctive, handmade, and classic gadgets, has
seen excessive progress during the last 5 years. Then the pandemic dramatically
modified buyers’ habits, resulting in extra shoppers buying on-line. As a
end result, the Etsy market grew from 45.7 million consumers on the finish of
2019 to 90.1 million consumers (97%) on the finish of 2021 and from 2.5 to five.3
million (112%) sellers in the identical interval.
The expansion massively elevated demand on the technical platform, scaling
visitors nearly 3X in a single day. And Etsy had signifcantly extra clients for
whom it wanted to proceed delivering nice experiences. To maintain up with
that demand, they needed to scale up infrastructure, product supply, and
expertise drastically. Whereas the expansion challenged groups, the enterprise was by no means
bottlenecked. Etsy’s groups have been in a position to ship new and improved
performance, and {the marketplace} continued to supply a wonderful buyer
expertise. This text and the subsequent kind the story of Etsy’s scaling technique.
Etsy’s foundational scaling work had began lengthy earlier than the pandemic. In
2017, Mike Fisher joined as CTO. Josh Silverman had not too long ago joined as Etsy’s
CEO, and was establishing institutional self-discipline to usher in a interval of
progress. Mike has a background in scaling high-growth corporations, and alongside
with Martin Abbott wrote a number of books on the subject, together with The Artwork of Scalability
and Scalability Guidelines.
Etsy relied on bodily {hardware} in two information facilities, presenting a number of
scaling challenges. With their anticipated progress, it was obvious that the
prices would ramp up shortly. It affected product groups’ agility as that they had
to plan far upfront for capability. As well as, the info facilities have been
based mostly in a single state, which represented an availability threat. It was clear
they wanted to maneuver onto the cloud shortly. After an evaluation, Mike and
his staff selected the Google Cloud Platform (GCP) because the cloud companion and
began to plan a program to maneuver their
many programs onto the cloud.
Whereas the cloud migration was taking place, Etsy was rising its enterprise and
its staff. Mike recognized the product supply course of as being one other
potential scaling bottleneck. The autonomy afforded to product groups had
prompted a problem: every staff was delivering in several methods. Becoming a member of a staff
meant studying a brand new set of practices, which was problematic as Etsy was
hiring many new individuals. As well as, that they had observed a number of product
initiatives that didn’t repay as anticipated. These indicators led management
to re-evaluate the effectiveness of their product planning and supply
processes.
Choosing a companion
Etsy had by no means labored with a software program improvement consultancy. To give you the option
to scale product supply and herald some outdoors experience, they began
on the lookout for a vendor. After an intensive search, they chose Thoughtworks,
primarily as a result of they perceived an in depth cultural match and deep alignment
concerning fashionable software program improvement.
Thoughtworks has the same method to Agile as Etsy – being Agile
quite than doing Agile. Etsy’s staff was not trying to undertake a
methodology that blindly follows strict rituals and practices, and wished
a companion that bases its method on agile rules and tradition.
Thoughtworks is thought for technical excellence and management in DevOps,
Steady Integration, and Steady Supply, all of which Etsy depends
on closely. As well as, the 2 corporations shared comparable rules
concerning worker care, variety, open-source, and expertise
sustainability.
The Thoughtworks staff began by embedding into product groups, working with
each purchaser and vendor groups, on varied important initiatives equivalent to cost
mannequin modifications, Search Engine Optimization (search engine optimisation), and a notification platform.
This enabled Thoughtworks to grasp how Etsy labored, created a right away
affect on supply, and improved improvement practices equivalent to automated
testing.
For Thoughtworks, Etsy was fairly completely different from the everyday consumer.
They’re a completely digital firm, and their technical course of and product
practises are very mature. Not like most purchasers who interact Thoughtworks,
Etsy had no massive transformation to undertake, and no ingrained habits
that wanted a serious shift. What they have been in search of was outdoors perspective
and supplementive abilities, to assist them incrementally enhance and sort out
their urgent initiatives. The tradition and rules have been already very
aligned with Thoughtworks’ personal, so the partnership grew to become extra of an
alternate of concepts drawn from completely different experiences.
Scaling Product Supply and Discovery
Thoughtworks and Etsy fashioned a cross-functional Product Supply
Tradition (PDC) staff to investigate the present product supply and discovery
course of. They analyzed your complete software program improvement worth stream,
trying intently on the complaints of managers and staff members. Chosen
occasions have been replayed to achieve a deep understanding of “how” it occurred
quite than who?. The Thoughtworks
staff, as a result of they hadn’t lived by the issues, have been in a position to convey
an goal outsider perspective.
There have been various issues Etsy was doing rather well:
- Cross-functional groups: They structured their groups round “4 desk legs”:
Product, Design, Engineering, and Analytics. All planning and supply practices
occur with collaboration among the many teams. - Incremental supply of worth to customers: Steady supply is the core
observe of their method. A code change sitting in a repository and never
deployed to customers is just not priceless, and successfully prices the corporate cash. Etsy
deploys a whole lot of instances a day, with a 30-minute decide to manufacturing lead
time. - Options are prioritized utilizing information: Product managers and information analysts
collaborate to find out an anticipated worth, measured by KPI modifications. If the
function doesn’t ship sufficient worth versus its value, as evidenced by an A/B
take a look at, it isn’t rolled out, and the code is eliminated.
The staff additionally uncovered alternatives for enchancment:
- Every staff was doing supply in several methods, having disparate processes
and rituals, making onboarding new staff and altering groups tough. Some
groups have been utilizing a course of like Scrum, whereas others didn’t use any formal
course of. - There have been various initiatives that had not created worth as
they anticipated. For instance, Etsy tried a present wrap function that had
decrease than anticipated adoption charges by sellers and consumers. - Previous to 2018, Etsy had been by a transitional interval, with
pivots in product technique and management modifications main some product
managers to expres a want for better possession over their product
selections.
Time to studying
The staff created an enchancment program, based mostly on lean considering and the
concepts of Marty Cagan, who had not too long ago offered at Etsy. The management staff was
studying his e-book Impressed.
I promise you that not less than half the concepts
in your roadmap usually are not going to ship what you hope. (By the way in which, the
actually good groups assume that not less than three quarters of the concepts received’t
carry out like they hope.)
After some preliminary analysis the staff got here up with a metric they referred to as Time
to Studying – the time it took for a product staff to validate an concept with a
buyer and achieve a greater understanding of its worth. They’d a baseline of
50 days that they wished to cut back.
They have been additionally different metrics.
- Enhancing the time to market
- Enhancing the effectiveness of merchandise (e.g. income, buyer
engagement) - Worker happiness
To affect the KPIs, they got here up with various answer hypotheses:
Lightweight prototypes that incorporate extra direct consumer analysis
Etsy has sturdy experimentation infrastructure and analytics capabilities.
They run many concurrent A/B exams on {the marketplace}. The issue the PDC
staff noticed is that the suggestions cycle could be fairly sluggish. For an concept to be
prepared for an A/B take a look at it needed to be close to production-level high quality. It additionally had
to have sufficient information to indicate statistical relevance. A/B exams would run for
months in lesser-used components of {the marketplace}.
To set your expectations, sturdy groups usually
take a look at many product concepts every week—on the order of 10 to twenty or extra per week. I
need to emphasize that these are experiments, usually run utilizing prototypes.
A prototype is just not one thing that’s prepared for prime time and positively not
one thing your organization would attempt to promote and stand behind. However they’re
immensely helpful, as they’re all about studying quick and low-cost.
To cut back the Time to Studying metric, the PDC staff utilized a lean UX
method. First, they began by shortly creating lo-fi prototypes and
instantly displaying them to customers. Second, the staff elevated consumer suggestions
classes to each two weeks and diminished the extent of ceremony. Third, they
created a dual-track system, with a design and analysis staff frequently
working forward of a supply staff. The concept was to effectively achieve confidence
in an concept earlier than it qualifies for an A/B take a look at. By lowering the price of
experimentation, they might improve the range and amount of experiments,
discovering concepts that confirmed extra worth.
Determine 1: Experiment validation course of
Blueprint for product supply
The PDC, along with product staff leaders, got here up with a set of
product supply rules and associated supply observe modifications. The
practices included: smaller extra targeted tales, redesigning story partitions to
higher match the staff course of, utilizing spikes to cut back technical uncertainty, and
higher methods of managing tech debt. They didn’t need to take away the autonomy of
groups. Steady enchancment is a giant a part of Etsy tradition. These practices
are supposed to be handled as smart defaults that may be tailored and improved
for every staff’s particular context.
The PDC staff first examined the observe modifications on pilot product groups to
set up their effectiveness after which iterated. As soon as the observe
demonstrated an enchancment, the staff added to a data base and supplied
mild teaching to introduce the brand new practices to all product groups throughout
Etsy.
Lean Portfolio administration
Some massive initiatives have been nonetheless being carried out, regardless that there was
not clear proof of the initiative creating worth. That is the everyday sunk
value downside. To resolve this, the PDC staff
urged taking a lean method to portfolio administration.
Our runway needs to be an inventory of hypotheses to check, not an inventory of
necessities to construct. After we reward our groups for his or her capacity to ship
necessities, it’s straightforward to quickly bloat our merchandise with pointless
options—resulting in elevated complexity, increased upkeep prices, and
restricted capacity to alter. Options delivered usually are not a measure of success,
enterprise outcomes are. Our runway is a collection of questions that we have to
take a look at to cut back uncertainty and enhance our understanding of progress
alternatives.
As an alternative of making prolonged venture specs and plans, the PDC staff
coached product groups to develop consequence statements for every initiative. With
consequence statements, beginning with a buyer or enterprise consequence permits the
staff to experiment with many alternative concepts to attain the end result by tying
them to measurable KPIs. The much less exact an consequence is, together with its KPIs,
the more durable it’s to specify. When getting ready for attainable initiatives, every
product staff created a two-page transient summarizing the end result, the measurable
affect, and the preliminary funding required.
The manager staff prioritized the end result statements, and every assertion
was seen as a “wager” amongst a portfolio. Impressed by the Lean Worth Tree
idea, the chief staff would observe a daily cadence to find out if a
wager was paying off, giving them the chance to pivot early.
Determine 2: An instance initiative assertion
Stronger product & engineering collaboration
One other enchancment was to incorporate engineers early within the product ideation
course of and to have them be part of consumer suggestions classes. The staff examined
engineers becoming a member of the speculation technology and validation classes and
working intently with designers throughout design sprints. The outcomes of the take a look at
confirmed a number of advantages. Engineers improved their understanding of the
buyer, constructing empathy for his or her wants. They might present on-the-spot
suggestions on the feasibility of an method and shortly create prototypes to
reveal concepts to customers. Cross-functional engagement allowed engineers to
contribute various nice concepts that made it to the product.
After the preliminary PDC initiative Etsy continued to strengthen the
collaboration between teams. At each degree of the group, product and
engineering are companions, which suggests they’re in a position to have wholesome trade-off
discussions. The platform group expanded using product managers for
infrastructure and developer tooling groups – encouraging groups to consider
technical capabilities as merchandise and end-users being Etsy engineers. The
technical product managers do analysis and prioritization actions, simply as
they might for a buyer product.
What was the affect of product supply tradition?
Testing with pilot groups indicated that the Time to Studying went from 50
days to five days. In the course of the pilot, one product supervisor mentioned, “What I really like about
the [direct user research sessions] pilot is that it lets us take a look at our concepts
quicker, it was very nice to not spend as a lot time diving deep into
one thing that we didn’t even know we have been going to make use of.” An initiative on
stock high quality that was predicted to ship a seven million greenback income
improve as an alternative delivered eleven million {dollars} in incremental worth utilizing
the PDC urged enhancements. Following this pilot success, the staff coached
your complete product improvement group on the product supply method.
In line with Mike Fisher, “It is extremely exhausting to isolate the advantages of a
course of change on a company of a number of hundred engineers since we don’t
run A/B experiments of this kind. Nevertheless, we did take a look at excessive degree
productiveness metrics after having PDC carried out for over 2 years, and all of
our metrics have been very constructive. We had improved our velocity by doing this
work, and we have been in a position to produce finally extra advantages for the consumers and
sellers – extra options, higher search, higher processes; the entire
market grew to become higher for consumers and sellers and that was the purpose.”
