By Sooraj Rajmohan
You are reading part two of a three-part series chronicling GOJEK’s entry into international markets, and the team that helped make it possible. If you haven’t read part one, get up to speed here:
Part 2/3 | Initiation
When we left Trifecta — the team responsible for charting GOJEK’s international expansion — in the last part of this series, the team was working on setting up a new environment for launch in Vietnam. But now, there were new problems to tackle.
Problem #1: Configuration drift
Until this point, GOJEK had always operated within Indonesia, but Trifecta was now in unmapped territory, building a completely new environment for GOJEK’s services in Vietnam. The services now needed to be configured separately across geographies, and there was no central location to manage these configurations effectively. This meant there was a chance of configuration inconsistencies creeping in over time (hence the term ‘drift’).
Problem #2: Deployment drift
Even if the team hacked a solution for the first problem, deploying the code would pose another one. It was imperative that the same version of code run on all the environments (the existing one in Indonesia, the one being built for Vietnam, and any future ones). Failing this, supporting services across countries would be a nightmare.
That wasn’t the kind of risk a startup that expounds lean engineering could afford to take.
Problem #3: A logging and monitoring solution for the Kubernetes cluster.
As this was the first time containers were being used by GOJEK, a new logging solution would be required for future debugging needs.
Much time had been lost already, it was time to bring in the big guns.
Even as the team dodged obstacles and took detours, making the best use of available resources, Project Managers Sourabh Gupta and Sanchit Juneja had been communicating with other teams for backup. They had found some ad-hoc support, but to see this journey through would need more seasoned soldiers.
Finally, this effort came to fruition.
Trifecta V2.0
After some negotiation, the team got the go-ahead for Sumit Gupta, another GOJEK veteran, to make the transition from GOJEK’s Atlas team to join Trifecta full time. Sumit had led GOJEK’s scaling efforts back in 2015, when the company was in constant firefighting mode, and knew a thing or two about dealing with the kind of situation the team was in. He would now assume the role of Trifecta’s Tech Lead.
With a renewed sense of purpose, the team looked back at the roadmap and reassessed priorities. This would be a challenging project. Daunting, yes. Insurmountable? Not quite.
If you’ve read GOJEK’s essential reading list, you may have noticed that The Mythical Man Month is one of the books we swear by, and one of its core tenets is: “Adding more people to a late software project makes it later.”
We added four more engineers to the team.
Dimensions and distant visions
GOJEK did not have previously devised solutions for the drifts it was dealing with. It needed new ones.
Tackling configuration drift.
It would now be Trifecta’s job to create a location within which services for different environments could be configured and accessed.
The solution was a service that used a new API to update configurations across geographies. These were then migrated to a configuration store.
This store was named Yggdrasil, the mythical Norse tree that connects the nine realms. Even in the frenzied sprint to accomplish a daunting task, we could never resist the temptation of poetic nomenclature.
Yggdrasil changed how configs were stored and managed in GOJEK.
Tackling deployment drift
Code.
Code fuels GOJEK, and ensuring that updated code is running in sync is one of the primary objectives of any team. Trifecta was no different — except for scale.
The team needed to figure out a way to ensure that the same version of code ran across multiple, disconnected environments. It was time to forge a new accessory.
A Palantir — a seeing stone.
When new versions of an application are released, there is a need to deploy the same version on all countries for the given environment. Palantir would take care of figuring out which countries the given deployment should go to.
The team also built a UI for Palantir, which gave a view of what version of a service was deployed in which cluster and sent alerts when there was a drift.
Tackling the logging problem
As launch date drew closer, this particular problem required non-linear thinking. Trifecta implemented an out-of-the-box solution using Stackdriver to unblock themselves for the launch. This was not optimal, but it provided enough breathing room to get through launch (post which specific tooling for logging on containers could be created).
Once again, the destination appeared near. With Yggdrasil keeping the services stable and Palantir keeping watch on code, GO-VIET was no longer a wistful dream, it was a visible apparition.
Run with Red
If you visit Indonesia, chances are you’ll see bright flecks of green amidst the traffic.
Green jackets, green helmets, green… everywhere. Green is now part of GOJEK’s identity in Indonesia, it is instant mental recall for the brand.
But this was not Indonesia.
GO-VIET, it was decided, would be born in red.
A local leadership team was put together in order to assemble other teams required for launch. This team was also flown to Jakarta to understand GOJEK’s values and how it works. These would serve as the core values for GO-VIET as well, with the leadership adapting things as required to suit Vietnam.
Meanwhile, Trifecta was stabilising the launch environment, collaborating with the newly-minted team in Vietnam, and working with multiple GOJEK teams.
By June of 2018, the backend was ready. A working driver and consumer app, as well as on-boarding portals for driver partners had been set up, along with crucial business-related teams like BI and Fraud. GO-VIET was taking shape right in front of its makers’ eyes.
Now for the testing:
Over the next one month, the app, and by extension, the new environment and assorted services, were put through the wringer. First, employees mocking as drivers tried it in a staging environment, then the actual on-boarded drivers were given a shot, so the team could record their feedback.
Testing then moved to the production environment. It was time for the real deal. The dry runs began, and in the final phase of testing, early adopters were given access to try out the alpha version of the app.
Finally, it was done. The environment, the services, the app, the operations, the tree, the stone… they had all survived.
GO-VIET was ready.
The final push
July 31, 2018 — Vietnam Beckoned
The environment was ready, everything was up and running. The app was live, with configurations changed so drivers would not be visible to anyone who downloaded it.
At 11.30 pm, the toggle was switched and test bookings were made on production with actual drivers and employees. The GO-VIET COO took the first ride. All was good.
Excitement was running high, and even GOJEK CEO Nadiem Makarim was in Ho-Chi Minh City (the first of GO-VIET’s operation centres) for the launch.
As the clock struck midnight, Trifecta hit the button, and specks of red lit up on the map within users’ GO-VIET apps. A few driver partners present at the launch venue confirmed that everything was working according to plan. It was done, we were live.
GOJEK had gone international.
Cheers, high fives, applause, congratulations, the pop of celebratory drinks.
Rumour has it, everyone present tried to guess how many orders GO-VIET would garner on its first day. Expectations were exceeded. The brand new service did >2,500 orders on day one, more than what GOJEK achieved on its first day in Indonesia.
It had been a long journey of planning and learning. The job was done; the path to Vietnam was charted. In the month of September, a formal launch was held, adding Hanoi to the list of cites the service operated in.
GOJEK, Indonesia’s first unicorn, had finally left home to find its place in the world. Now, all eyes turned to Thailand. Read what awaited us there in the part three of this series.
When you’re building a Super App for Southeast Asia, coming up against tough problems is common. Solving them, however, takes uncommon skill. That’s what makes GOJEK an engineer’s playground. If you’ve got the game, check out gojek.jobs to see our available openings. See you on the other side. ✌️
