Skip to content

High-Tech Solutions

Utilizing Golang to Build a Product Search Engine for a Global Tech Leader’s CPQ Portal

Utilizing Golang to Build a Product Search Engine for a Global Tech Leader’s CPQ Portal

Success Story

In 2020, a worldwide tech giant reached out to Scalefocus to help them design and implement a next-generation product search engine that would replace the product finder functionality available on their existing CPQ platform. The legacy solution relied on a relational database, Java-based backend monolith, and JSP front end. Unfortunately, but not unexpectedly, it delivered inferior user experience and search performance.

As part of a significant CPQ transformation program, which also aimed at operational cost reduction and exiting a license-heavy proprietary software commitment, Scalefocus took the product search functionality to another level by leveraging top-notch cloud technologies employed in a microservice architecture solution.

25x faster search
result response time from 10 sec to under 200 ms

35% reduction in infrastructure and maintenance operational cost

40% improved
product data load
and data indexing

The Client

A multinational technology company that specializes in a wide range of hardware and software products and services. Over the years, it has grown into one of the world’s leading technology companies, and it is known for its innovative products, global reach, and commitment to sustainability.

The Challenge

  • Legacy System Transformation  

The tech giant’s CPQ (Configure, Price, Quote) portal relied on a legacy product finder functionality that was outdated enough not to meet modern e-commerce standards. Transforming this legacy system into a dynamic, e-commerce-like product search engine was among the most daunting tasks. 

  • Feature-rich Functionality 

The new system needed to support advanced features such as search-as-you-type, infinity scroll, guided selling, and auto-attach of related products. Ensuring smooth integration of these features while maintaining high performance and responsiveness enhanced the project’s complexity. 

  • Performance Requirements 

Scalefocus had to make sure that the search result response times were below 200ms. This posed another significant challenge, especially with the scale and complexity of the product catalog. 

  • Costs Optimization 

The client sought to optimize operational costs compared to the legacy, license-heavy proprietary CPQ solution. This necessitated a cost-effective and scalable solution leveraging cloud-native technologies. 

The Solution

  • Adoption of Go Programming Language 

We implemented backend services using the Go programming language to address the above challenges. Go’s lightweight concurrency model and efficient runtime made it well-suited for handling the search engine’s concurrent, high-throughput requirements. 

  • Integration with OpenSearch 

Go’s extensive ecosystem and world-class support for concurrent programming enabled seamless integration with OpenSearch, the chosen search engine technology. Go’s native support for simultaneous operations allowed for parallel indexing and querying large datasets, improving search performance and response times. Its robust standard library and support for HTTP made it an ideal choice for interacting with OpenSearch, ensuring seamless integration and efficient handling of search queries. 

  • GraphQL and gRPC 

Employing GraphQL for an API layer and gRPC for internal communication between microservices further enhanced performance and flexibility. Go’s native support for both technologies simplified integration and reduced development overhead. 

  • Optimized Cloud Deployment 

Leveraging AWS cloud services and scalable and cost-effective infrastructure facilitated the deployment of Go-based backend services. AWS’s auto-scaling capabilities allowed the system to adjust resources dynamically based on demand. This ensured optimal performance during peak usage while minimizing operational costs during off-peak times. Go’s efficient resource utilization and scalability further contributed to lower operational costs. 

  • React for Front End 

Utilizing React for the front end enabled the development of a dynamic and responsive user interface, complementing the advanced search functionalities. 

  • Terraform for AWS Automation 

Terraform simplified the deployment and management of AWS cloud infrastructure, ensuring scalability, reliability, and cost-effectiveness.

The Project 

After an initial phase of analyzing the legacy solution and understanding and clarifying all initial requirements, our team came up with an entire solution architecture proposal. Our experts divided the functional data flows into Search and Product Data Ingestion. The search flow comprises a modern user interface with responsive design and user experience implemented in React, employing the MUI lib. The UI interacts with a GraphQL middleware API to reach out to the product search-related microservices of the backend implemented in GO. The GO microservices interact directly with the AWS-hosted OpenSearch cluster to execute lightning-fast product search queries. The product data ingestion-related microservices are also implemented in GO. Product data is ingested from three different sources and pushed into the OpenSearch cluster in a data model designed for optimal product data analysis and indexation. The primary product data source is a relational database, the source of truth for product details and product relationships. A second data source provides regular pricing information over an SQS Queue, and the third delivers product stock availability data over a file share. 

Scalefocus chose GO for the server side because of its velocity and ability to run on the operating system layer without a virtual machine or interpreter. We built the majority of concurrency logic using GO because of its simplicity and natural feel when constructing the parallelism.  

We chose GraphQL for front-end API technology for its ability to reduce the number of HTTP requests significantly and solve the issues with over-fetching and under-fetching. We also chose gRPC over the REST server layer as gRPC is much more compact, not to mention it is roughly 7 times faster than REST when receiving data and 10 times faster when sending data for a particular payload. Last but not least, GO’s native support for both GraphQL and gRPC simplified integration and significantly reduced development overhead

Business Results 

  • 25 Times Faster Performance and Responsiveness 

By leveraging Go’s concurrency features and optimizing the deployment on the AWS cloud, the search engine achieved remarkable performance gains. The search result response time was consistently kept under 200 milliseconds, down from 5 to 10 seconds. This was critical for meeting the stringent performance requirements and delivering a seamless browsing experience to users. The performance of product data load and data indexing flows also improved by 40 percent.  

  • 35% Reduction in Operational Costs

The efficient utilization of AWS cloud resources, combined with Go’s lightweight runtime, helped reduce operational costs associated with infrastructure provisioning and maintenance by an impressive 35 percent. The auto-scaling capabilities of AWS ensured that resources could be scaled up or down based on demand to optimize cost-efficiency without sacrificing performance. 

  • Enhanced User Experience 

Implementing the e-commerce-like product search engine significantly improved the user experience of the CPQ portal. Advanced features such as search-as-you-type and guided selling provided users with intuitive and efficient ways to discover products, increasing user satisfaction and engagement.

Technologies

Golang
AWS
OpenSearch
GraphQL
GRPC
React
Terraform

Our Work

We have a global client base that includes Fortune 500 companies, innovative startups and industry leaders in Information Technology, E-Commerce, Insurance, Healthcare, Finance and Energy & Utilities.

Explore all

Ready to scale and meet the technology challenges of tomorrow?

Contact us