The Events Wave: Embracing Real-Time Agility with Event-Driven Architecture at Thoughtworks
Abstract
Event-driven architecture (EDA) offers a strategic business advantage, demanding a growth mindset for successful implementation. The use of EDA mechanisms and near-real-time integrations provides unspoken synergistic benefits for businesses.
As a global technology consultant, Thoughtworks embraced a Digital Platform Strategy in 2017, positioning itself as a front runner. The adoption of EDA became pivotal for continuous system iteration, ensuring real-time data.
Our former EDA enablement platform, the ‘Events Platform’ was developed with Redis and RabbitMQ. The challenge was not the technology choice but the platform’s stagnation and growing scaling needs of ThoughtWorks.
After evaluating messaging platforms, Kafka was chosen for its core capabilities in scaling, availability, and reliability. The Events Platform’s capabilities were re-implemented on Kafka, creating a robust product.
Validation occurred through a pilot and observation period, testing events in the new platform with satisfactory results. Migrating the Events Platform followed a meticulously planned strategy, allowing seamless transition while event-driven remaining active.
In the second phase, we integrated the Events Platform into our award-winning Internal Developer Platform (IDP), NEO. This facilitated a seamless experience for developers to publish and consume events.
With the core events platform in NEO, we delivered sought-after features, such as attribute value-based event authorization, adopting cloud events specifications for interoperability, and encrypting event payloads at rest.
In conclusion, the Events Platform, built on Confluent Kafka, delivered substantial business value for Thoughtworks, establishing resilient systems and seamless interactions. Our aim is to share this success story, providing insights into the adoption mindset and guiding enterprises in building scalable and robust systems using Kafka.
As a global technology consultant, Thoughtworks embraced a Digital Platform Strategy in 2017, positioning itself as a front runner. The adoption of EDA became pivotal for continuous system iteration, ensuring real-time data.
Our former EDA enablement platform, the ‘Events Platform’ was developed with Redis and RabbitMQ. The challenge was not the technology choice but the platform’s stagnation and growing scaling needs of ThoughtWorks.
After evaluating messaging platforms, Kafka was chosen for its core capabilities in scaling, availability, and reliability. The Events Platform’s capabilities were re-implemented on Kafka, creating a robust product.
Validation occurred through a pilot and observation period, testing events in the new platform with satisfactory results. Migrating the Events Platform followed a meticulously planned strategy, allowing seamless transition while event-driven remaining active.
In the second phase, we integrated the Events Platform into our award-winning Internal Developer Platform (IDP), NEO. This facilitated a seamless experience for developers to publish and consume events.
With the core events platform in NEO, we delivered sought-after features, such as attribute value-based event authorization, adopting cloud events specifications for interoperability, and encrypting event payloads at rest.
In conclusion, the Events Platform, built on Confluent Kafka, delivered substantial business value for Thoughtworks, establishing resilient systems and seamless interactions. Our aim is to share this success story, providing insights into the adoption mindset and guiding enterprises in building scalable and robust systems using Kafka.
Date
May 2, 2024 5:00 PM
Event
Location
Bangalore
Sheraton Grand Bengaluru Whitefield Hotel & Convention Center, Bengaluru,
