Reader’s Note: This post references CockroachDB Serverless and/or CockroachDB Dedicated which, as of September 26 2024, have been renamed and incorporated within the new CockroachDB Cloud platform, which you can read more about here.
This article is co-authored by Harsh Shah, Staff Sales Engineer for Cockroach Labs, and Brian Lee, VP of Engineering at Nellis Auction.
Nellis Auction, an online auction company based in Las Vegas, NV, specializes in reverse logistics – this means handling goods that travel back through the supply chain from customer to distributor or manufacturer, then selling them off in local markets via auction.
With the company's rapid growth, particularly in warehouse operations, scalability, and service distribution, they faced new challenges in delivering highly reliable services under load. This expansion led Nellis Auction to seek a robust data solution, eventually partnering with Google Cloud (GC) and CockroachDB.
The Transformation Journey
Three years ago, Nellis Auction embarked on a transformative journey, moving away from reliance on external parties for development services and hosting. At this stage, the company migrated its legacy MySQL 5.7 database to Cloud SQL on Google Cloud.
While this provided better reliability and performance, it was a temporary solution that lacked modern SQL database features and effective distribution strategies. Limited support for Cloud SQL compounded their problems, leading to the exploration of more robust data solutions.
Nellis Auction encountered several significant challenges as they initially expanded their operations:
Scalability and Reliability Issues: Their legacy MySQL 5.7 database on Cloud SQL was not built for long-term scalability, lacking modern features necessary to support growing data needs and increasing operational complexity.
Limited Support: As operations scaled, traffic spikes and anomalies highlighted the inadequacies in their support strategy for Cloud SQL, leading to delays and potential service disruptions without timely assistance.
Replication Complexity: Introducing a read-only replica to manage high traffic proved complex and problematic, with misconfigurations causing more harm than benefit, forcing them to seek more reliable solutions.
The absence of advanced distribution and redundancy capabilities in their legacy system drove the search for a new database solution that could support their expanding business.
Why CockroachDB: Reliability Required
Nellis Auction learned about CockroachDB during a period of significant transformation, as they sought to upgrade their infrastructure and move away from reliance on external parties for development services. While migrating their legacy MySQL database to Cloud SQL on Google Cloud Platform provided a temporary solution, they faced challenges with distribution strategies and modern SQL features. In their search for a reliable, modern database solution, CockroachDB emerged as the only option that met all their requirements:
Natively Distributed and Fault Tolerant: CockroachDB's native replication and redundancy capabilities suited Nellis Auction's needs for regional replication and survivability without considering these features as afterthoughts.
SQL Compatibility: SQL was integral to both the application stack and the business operations at Nellis Auction. CockroachDB’s adherence to SQL avoided the need for the company to invest in learning a new language and allowed them to leverage existing SQL knowledge.
Relational Transactional Isolation: Maintaining a relational data storage medium with highly reliable isolated transactions was crucial for Nellis Auction’s architecture.
Open and Portable: CockroachDB's cloud-agnostic nature, fully managed cloud offering, and availability via Docker/Kubernetes provided significant flexibility. Its open-source nature decreased risk and facilitated easy development environment setups and migration handling.
Industry Experience and Product Focus: CockroachDB was built by industry veterans who understood the pain points of scaling a data layer. The company’s singular focus on delivering an advanced SQL database designed for modern applications provided the confidence needed to adopt CockroachDB.
Specialized Support: CockroachDB’s dedicated support team, knowledgeable about the product from development to deployment, offered unmatched assistance that proved crucial for a significant database migration.
Serverless Offerings: CockroachDB's serverless product was perfect for smaller, bespoke applications within Nellis Auction, providing a pay-as-you-go model while reusing code from other applications.
Data Architecture and Integration: Strategic Choices
Database migrations can be challenging, often causing downtime or data consistency issues. Nellis Auction made strategic choices to adopt CockroachDB within their GC infrastructure, leveraging various GC products and CockroachDB features. Here’s a succinct breakdown of their architecture and integration:
Google Cloud Run: This serverless application hosting service is used for running highly scalable web services and individual jobs. It plays a crucial role in data replication across services and databases, providing the flexibility to scale operations as needed.
CockroachDB: At the heart of Nellis Auction's data strategy, CockroachDB is used for its natively distributed, fault-tolerant architecture. It supports SQL, ensuring compatibility with existing applications and knowledge. CockroachDB also enables seamless scaling and enables data consistency across multiple regions, supporting the company's growth and operational demands.
Google BigQuery: Acting as the aggregation point for operational data, BigQuery collects data from both legacy Cloud SQL databases and new CockroachDB clusters. This facilitates seamless business continuity and allows for comprehensive data analysis without impacting the performance of the operational databases.
Google Datastream: To maintain a constant feed of data from Cloud SQL to BigQuery, Nellis Auction uses Datastream. This service enables continuous and real-time data replication with minimal effort, ensuring that both historical and current data are available for analysis.
Google Looker: After aggregating data in BigQuery, Looker is used to deliver insights to various departments. This tool allows different teams to create and share dashboards, helping them make data-driven decisions and run more efficiently.
Google Pub/Sub: Serving as the primary message bus, Pub/Sub transmits data between services, databases, and BigQuery. It allows reliable data delivery and integration across the entire infrastructure, facilitating smooth communication between different components.
CockroachDB Changefeeds: One of the standout features of CockroachDB is its native support for changefeeds. These feeds are integrated directly with Google Pub/Sub, allowing real-time data changes to be sent to various components such as application services and BigQuery for visibility.
Google MemoryStore: For lightweight and less critical message delivery, MemoryStore is used. It provides a simple and efficient way to transmit messages across service boundaries, reducing the load on relational data stores and improving overall system performance.
Temporal: This orchestration service is essential for scenarios requiring reliable and bi-directional message delivery across services. Temporal ensures that complex workflows are handled smoothly, guaranteeing data consistency and reliability in multi-step processes.
By leveraging these tools and services, Nellis Auction has created a robust and scalable infrastructure that supports their rapid growth and complex operational needs. This architecture not only improves performance and reliability but also provides the flexibility to adapt to future requirements.
The Right Database for Rapid Growth
Nellis Auction’s decision to partner with GC and CockroachDB was driven by the need for a reliable, scalable, and modern data solution. CockroachDB’s distributed nature, SQL compatibility, relational transactional isolation, flexibility, and robust support made it the perfect choice to support their rapid growth and complex operational requirements.
Learn more about how global enterprises rely on CockroachDB to drive business growth and meet customer expectations. Get the guide! From Resilience to Scalability: 12 Mission-Critical CockroachDB Use Cases.
