After Fleeing Vietnam, This Family Transformed a Neighborhood Bakery Into a Leading Manufacturer of Baked Goods

After Fleeing Vietnam, This Family Transformed a Neighborhood Bakery Into a Leading Manufacturer of Baked Goods

By Ian McCue, reporter at Grow Wire
8-minute read

In short:

  • The Ly family endured a harrowing journey from Vietnam to Malaysia in 1978, then spent time in a refugee camp before eventually immigrating to the U.S.
  • Six years later, the five Ly brothers eventually pooled together their money to purchase a small bakery in San Francisco.
  • Sugar Bowl Bakery has grown into an extremely successful producer of baked goods, with its products available in Costco, Safeway and Walmart.

On Oct. 15, 1978, 148 people squeezed onto a 45-foot, single-engine fishing boat and started the treacherous odyssey across the South China Sea. They fled South Vietnam, which the Communist forces of North Vietnam had captured a few years earlier, and headed south to Malaysia in search of a new beginning.

Among the passengers were the Ly family, including Andrew Ly, his parents, his sister and her family and two of his brothers and their families. This was their fourth attempt to escape Vietnam since the fall of Saigon in 1975. Their ultimate goal was to make it to the United States, with its great promise of opportunity. They could’ve only guessed that one day, they would indeed settle there and build a thriving commercial bakery.

This was the Ly family's fourth attempt to escape Vietnam since the fall of Saigon. They could’ve only guessed that one day, they would settle in the U.S. and build a commercial bakery.

During the six-day journey, Thai pirates attacked the Lys’ boat three times, taking money and any other possessions they could find. Along the route, the Lys saw capsized boats just like their own and fellow refugees who had gone down with them.

“We were lucky that the ocean was very friendly at the time we left,” Andrew, now in his late 60s, said.

When the family and their fellow passengers reached Malaysia, officials first turned them away. After spending another day at sea, the refugees tried again, sinking their boat upon reaching the beach so they had no way out. This time, reporters from the Far Eastern Economic Review happened to be standing nearby, and their presence prevented Malaysian officials from rejecting the refugees. The incident was a lucky break on the Lys’ long migration from Vietnam to the U.S.

The Ly family fled Vietnam in 1978. Their arrival in Malaysia was on the cover of the Far Eastern Economic Review.

The Lys settled at Malaysia’s Pulau Bidong camp. Nine months later, Andrew and his parents were able to immigrate to San Francisco, where his oldest brother had recently moved, while the others stayed behind.

The first bakery

Andrew and his siblings, who over time all made it to the Bay Area, took whatever jobs they could to help cover the bills – delivering newspapers, washing dishes, janitorial work and construction. Their other family members later joined them in the Bay Area, and they all saved as much money as they could.

By 1984, four years after arriving in the U.S., the five Ly brothers had collectively saved $40,000. They heard through a friend that the owners of a small coffee shop in San Francisco’s Richmond District called Sugar Bowl Bakery wanted to sell their store and the Lys decided to purchase it.

“At the time we came here, we didn’t have any special skills, we did not have the ability to speak English well,” Andrew said. “So basically donut and coffee shops we thought was the easiest business for us to run.”

After saving their income for four years, the Ly brothers purchased a small coffee shop in San Francisco in 1984.

The previous owners taught the recent immigrants how to make donuts and popular French pastries. Sugar Bowl’s revenue nearly doubled to around $200,000 in the first year the Ly family owned it, a spike driven by introducing new items, improving the efficiency of store operations and selling quality pastries for a fair price. Much of that quality came from the fact that the family used real butter in its baked goods while many competitors substituted cheaper alternatives like margarine or shortening.

By 1986, the Lys had enough money to purchase a second retail store a few miles away. A few of Andrew’s brothers wanted to put that money toward an apartment instead, but he convinced them a second business location was a better investment. Making critical decisions without the consensus of the family would become a theme over the years, especially after Andrew graduated from San Francisco State in 1986 with an accounting degree and took on more of a leadership role in the business.

While Andrew admits that can be challenging, he’s learned to shoulder the full responsibility for decisions he is confident are in the family’s best interests.

Exploring new channels

For years, Andrew attended industry and networking events, slowly earning the trust of people across the San Francisco food industry. Through those relationships, he learned that hotels and other businesses wanted to outsource their bakery operations and identified a key opportunity the early ‘90s: food service. With the help of a Small Business Administration (SBA) loan, Sugar Bowl purchased a manufacturing facility in San Francisco to handle larger-scale production. Sugar Bowl started making baked goods for local cafes, small markets and convenience stores. But within a few years, it counted major hotel chains, the Moscone Convention Center and large tech companies as customers.

Through networking, Andrew identified a key opportunity the early ‘90s: food service.

But Sugar Bowl’s biggest break came about a decade later. After years of trying to sell through Costco, the bakery hired a food broker who previously worked for the retailer to get its foot in the door. Buyers eventually toured Sugar Bowl’s factory, where palmiers caught their eyes. Palmiers would be Sugar Bowl’s first item sold at the warehouse club.

As a commercial bakery, Sugar Bowl makes items including palmier cookies for retailers like Costco.

Even as one of the largest retailers in the country, Costco often works with small vendors and allowed the family-owned business to start with just a handful of stores in Northern California. If the products sold well, Sugar Bowl could scale its manufacturing operations as more Costco stores sold its sweet treats. The visibility Costco provided helped the bakery convince Safeway to stock its products soon after.

sign up for the growwire newslettersign up for the growwire newsletter

Around that same time, Michael Ly, Andrew’s nephew and the son of one of Sugar Bowl’s co-founders, left his job at a biometrics software company in Los Angeles to join the family business. Michael, who was just 4 years old when he fled Vietnam with his parents and Andrew, spent many hours of his childhood at the bakery, walking there after school to do his homework and helping clean up at closing time.

Around that time, Michael Ly, Andrew’s nephew and the son of one of Sugar Bowl’s co-founders, left his job to join the family business.  

“I could see that there was a lot more opportunity for the business, then Andrew needed more help at that time so he did ask me [to return],” said Michael, today Sugar Bowl’s vice president and general manager.

Sugar Bowl eventually expanded to seven retail stores. It developed a loyal customer base by selling tasty French pastries at a reasonable price. It could keep prices down thanks to the economies of scale created by buying ingredients in bulk and automating production. Michael draws a parallel between Sugar Bowl and Starbucks, which introduced many Americans to Italian coffee drinks like lattes and espresso.

“Starbucks didn’t invent it, they brought it from Italy,” he said. “But they drove the price down enough and they made it accessible and they found a market that a lot of people would pay 3-4 times more for a soy latte vs. just a regular black Folger’s coffee at a donut shop.”

Michael draws a parallel between Sugar Bowl and Starbucks: “Starbucks didn’t invent it, they brought it from Italy,” he said. “But they drove the price down enough and they made it accessible."  

That rapid expansion required a lot of capital, and Andrew said financing has consistently been the company’s biggest challenge. Sugar Bowl struggled to secure lenders in its early days -- the sellers themselves provided the Lys a loan for their second store. Aside from the SBA loan, Sugar Bowl is primarily self-funded, pumping much of its profits back into the business.

Going all in on manufacturing

In 2010, Sugar Bowl made the tough decision to sell off its food-service division and sell or close all of its storefronts to focus exclusively on manufacturing for other retailers. Andrew’s brothers, who managed the various cafes, became less involved with the business at that point, though they are still on the board of directors and shareholders.

“It was a long process, it was a tedious process,” Andrew said. “It was a difficult decision to close all the retail shops because we started from there, it was difficult to let it go. But we knew it was better for the family and better for my brothers when they grew a little older.”

The five Ly brothers started Sugar Bowl together. Now, Andrew (center) serves as CEO.

Under this new model, Sugar Bowl could concentrate on a few key products and ship them out by the truckload instead of baking more than 400 different products for hundreds of small orders. Today, Sugar Bowl has just five core products: madeleines, palmiers, brownie bites, duets (half madeleine, half brownie bite) and apple fritters. It chose those products not because they were easy to make in bulk but because after analyzing sales data, it was clear this handful of items accounted for a disproportionate amount of revenue. (Think the Pareto Principle.)

The company’s R&D department is always testing new products, some of which make it onto store shelves. But it’s found tremendous success with those five sweets.

“I think most consumers, they eat the same things mostly every month,” Michael said. “They don’t venture off and try too many different things on a yearly basis. You want to be one of those things, but it’s not like they’re going to try 20 different things this year.”

Sugar Bowl's most popular treats include French madeleine cookies.

Sugar Bowl could scale the business quickly because it purchased two additional, large manufacturing facilities – each about 55,000 square feet – across the San Francisco Bay in Hayward, Calif. It has added more automation equipment over the years to produce about 2-3 million pieces per day, twice what its daily production was five years ago.

Today, Sugar Bowl’s pastries can be found in a who’s who of retailers: Walmart, Target, Kroger and Smart & Final. It’s mostly up to retailers whether they sell those goods under their private label or the Sugar Bowl brand.

A new frontier

While Andrew remains Sugar Bowl’s full-time CEO, he has started to pass on the reins to the next generation of the Ly family and executives outside it. Andrew describes his most important duties today as being “the ambassador for the company” and “creating the culture” at what has blossomed into one of the country’s largest privately-owned bakeries, with about 400 employees.

Sugar Bowl has blossomed into one of the country’s largest privately-owned bakeries, with about 400 employees.

The bakery is battling the rising cost of labor in the Bay Area, due to low unemployment and the highest cost of living in the U.S., but it shows no signs of slowing down as it begins its next act. Sugar Bowl is trying to develop a stronger foothold in the Midwest and Eastern U.S. by partnering with regional grocers like H-E-B and Publix. That initiative will be key to reaching its goal of doubling sales in the next five years. (For the last several years, revenue has increased 10-15% annually.)

While international sales are not a major source of revenue, Sugar Bowl products are available in Mexico and the Caribbean, and this year it started selling into Australia and China. Andrew said he hopes Sugar Bowl grows into "one of the major sweet baked goods companies in the world.”

“The key for us is I think we have a product line that could have mainstream appeal,” Michael said. “It’s not like it’s a very specific kind of need. I think we have very simple flavors that cut across different cultures and different countries.”

Sugar Bowl plans to grow its commercial baking business in the midwestern U.S. and abroad.

Earning presidential recognition

Many laud Sugar Bowl as a tremendous success story, especially given the Ly family’s struggle to get to the U.S. In 2013, then-President Barack Obama called the company out during a speech on immigration reform in San Francisco. He pointed to the Lys as a gleaming example of what immigrants can accomplish in America.

It was the ultimate validation. The story of the Ly family had come full circle.

“We picked America because America … had the best opportunity for success,” Michael said. “It was the most respected, the most admired country. But we didn’t expect that landing in America, you were going to get all this success handed to you. We just wanted the opportunity.

“When the President of the United States brings you up, of course you feel immense pride. He’s calling Andrew by name and our family as an example of the American Dream. It’s hard not to feel very proud and grateful for what you have accomplished since immigrating.”