In the dynamic world of online sports betting and iGaming, Hard Rock Digital has emerged as a trailblazer, seamlessly blending entertainment with cutting-edge technology. Their journey to create a robust, compliant, and user-friendly platform is a testament to innovation and strategic collaboration.
Navigating the Regulatory Labyrinth
The U.S. sports betting landscape is a complex web of federal and state regulations. The Federal Wire Act mandates that betting transactions occur within the state where the bet is placed, prohibiting interstate wagering. Additionally, each state enforces its own set of rules, further complicating the operational framework for online betting platforms.
Initially, Hard Rock Digital considered deploying separate databases for each state to ensure compliance. However, this approach would have led to exorbitant costs and operational inefficiencies along with challenges associated with resilience at scale. The challenge was clear: how to maintain a unified platform that adheres to diverse regulatory requirements without compromising performance or user experience.
Rethinking Hybrid with CockroachDB & AWS
Enter CockroachDB, a distributed SQL database that runs anywhere with consistency, resilience and scalability. By adopting CockroachDB, Hard Rock Digital established a single logical database capable of operating across multiple jurisdictions. For instance, in states with stringent data location requirements, Hard Rock Digital utilizes AWS Outposts to extend AWS infrastructure into local data centers.
This setup enables the deployment of CockroachDB nodes within the required jurisdictions like Florida, Arizona, Indiana, New Jersey, Ohio, Tennessee, Virginia, and Illinois, effectively addressing compliance mandates while optimizing infrastructure costs. This architecture also unlocks data residency requirements and compliance with local regulations.
But, you may ask what AWS Outposts is and why CockroachDB was truly the solution that many customers including Hard Rock Digital have chosen in their solution stack.
“Instead of many databases spread across the United States, we have one logical database, one logical system, one thing that we’re managing and running regardless of the jurisdiction,” said Senior Platform Architect Joe Rizzo. “Instead of investing all the capital to manage and monitor and run these disparate database installations, we can just collapse everything into a single CockroachDB cluster. And then distribute the nodes to the geographic locations where they’re needed to meet the residency requirements, the data residency and compute requirements.”
AWS Outposts is a fully managed service that extends the same AWS infrastructure, services, APIs, and tools to virtually any datacenter, co-location space, or on-premises facility. The service delivers AWS infrastructure and services to on-premises locations in more than 50 countries and territories, enabling customers to run AWS compute, storage, database, and other services on-premises while seamlessly connecting to AWS' broad global infrastructure.
CockroachDB is a distributed SQL database that can easily run on the EC2 instances running on AWS Outposts to achieve low-latency communication and leverage its cloud-native features in their hybrid environments. CockroachDB is designed to build, scale, and manage cloud services in diverse environments with data stored globally, offering several key features that are valuable:
Distributed SQL: Scale data horizontally across servers, regions, and continents. Data is automatically replicated and redistributed to optimize performance and resilience without manual sharding.
Transactional Consistency: Data remains correct and up-to-date across all nodes in the cluster, regardless of physical location, ensuring data integrity and reliability.
High Availability: Data is automatically replicated across multiple nodes, ensuring that if one node goes down, CockroachDB continues to function, minimizing downtime and maximizing availability.
Compatibility: CockroachDB uses standard SQL syntax that is wire-compatible with PostgreSQL, allowing customers to leverage existing PostgreSQL client libraries and tools, simplifying adoption and integration.
Cloud-Native: Operates seamlessly across public, private, and hybrid cloud environments, as well as on-premises deployments. Its single executable can run on EC2 or EKS, scaling out to leverage cloud elasticity.
We will now cover some key lessons learned from the front lines building out this hybrid solution for Hard Rock Digital.
Lessons from the Frontlines
The journey was not without its learning curves. We will cover some key considerations and some best practices you should be aware of when using AWS outposts with CockroachDB.
High availability & fault tolerance
To ensure high availability and fault tolerance, it is recommended to deploy multiple CockroachDB instances locally on AWS Outposts. By utilizing placement groups, you can strategically distribute these instances across the underlying hardware resources on the Outpost.
AWS Outposts allows you to create and use placement groups in the same way as you would in regular Availability Zones. When creating a placement group with a spread strategy on an Outpost, you can choose to spread instances across hosts or racks. Users can also inform CockroachDB nodes what rack they are setting up for each node allowing CockroachDB to influence diverse replica placements.
Spreading instances across hosts enables you to implement a spread strategy even on a single-rack Outpost, ensuring that instances are distributed across different physical hardware for increased resiliency.
Hybrid deployment network connectivity
When you need to establish communication between a database hosted on an AWS Outpost and another database in an AWS Region, it's recommended to leverage AWS Direct Connect in conjunction with a Local Gateway.
This architecture ensures optimal performance and security for your data transfer while preserving the Service Link bandwidth for management tasks. The Service Link is a necessary connection between your Outposts and the AWS Region (or home region).
It allows you to launch Amazon EC2 instances, attach Amazon EBS volumes, and access AWS services such as Amazon EKS, Amazon EMR, and CloudWatch metrics with optimal performance. However, for high-throughput data transfer between your on-premises Outposts infrastructure and AWS services in the Region, it's advisable to establish a private connection using AWS Direct Connect and a Local Gateway.
By using a Local Gateway with Direct Connect, you can create a dedicated, high-bandwidth, and low-latency connection between your Outposts and the AWS Region.
This architecture ensures optimal performance for your database communication while maintaining the Service Link bandwidth for management tasks, such as software updates, monitoring, and control plane operations.
Load Balancer Configuration
When configuring your Network Load Balancer (NLB) setup, it's recommended to use separate target groups for your database port traffic and CockroachDB web console port traffic. By doing so, you ensure that the unavailability of the web console instance(s) does not impact the database traffic routing.
This separation of concerns enhances the overall availability and resilience of your architecture. If you choose to use a single NLB to host both types of traffic, it's crucial to configure the
target_group_health.unhealthy_state_routing.minimum_healthy_targets.countattribute appropriately for the web console target group.Setting this attribute to off for the web console target group ensures that even if all instances in that target group become unhealthy, the NLB will continue routing traffic to the remaining healthy instances in the database target group. By following this approach, you effectively decouple the availability of your web console from your database, minimizing the risk of cascading failures and improving the overall resilience of your application.
Storage Configuration
When configuring Amazon EC2 instances, if using gp3 general purpose Amazon EBS storage, consider using multiple EBS volumes and the backup Write Ahead Log (WAL) failover feature of CockroachDB in order to improve resiliency of the cluster in the case of a disk stall.
A Harmonious Fusion of Entertainment and Technology
“CockroachDB is at the core. That's life, that’s the blood, that’s the crown jewels. Our application runs on top of Cockroach, and then underneath the compute runs on AWS Cloud Continuum.”
By leveraging CockroachDB's capabilities, Hard Rock Digital has crafted a platform that not only meets stringent regulatory standards but also delivers a seamless and engaging user experience. This strategic approach has positioned them as a leader in the online sports betting and iGaming industry, setting a benchmark for innovation and compliance.
In the ever-evolving digital landscape, Hard Rock Digital's story is a compelling example of how thoughtful integration of technology and strategic planning can overcome complex challenges, paving the way for future advancements in the industry.
Interested in learning more about CockroachDB on AWS? Check out our webpage here or start instantly with CockroachDB on AWS Marketplace.
Hard Rock Digital + CockroachDB at AWS re:Invent 2024
Cockroach Labs is coming to AWS re:Invent 2024! In our session, “DAT336-S | How Hard Rock Digital built a multi-region sportsbook with CockroachDB,” you’ll hear from James Lupolt, Database Engineer at Hard Rock Digital, and Rob Reid, Technical Evangelist at Cockroach Labs. The duo will discuss how the Hard Rock Digital team launched their betting app with CockroachDB, while navigating strict regulatory requirements. In addition, the app needed to be highly available, able to manage high transaction loads that can be spiky, and be performant at scale across multiple regions to accommodate Hard Rock’s global audience. Check out our session to learn how Hard Rock Digital uses Amazon EC2 and AWS Outposts to run CockroachDB Cloud on AWS.
The session will be held on Monday, December 2 at 2:30-3:30 PM PST. Find us at the Wynn Hotel, Level 1, Lafite 4, Content Hub, Turquoise Screen. If you can’t make the session, check us out at Booth #444 of the Expo Hall! We hope to see you there!
